Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Does Process.WaitForExit() Really Wait?

106 views
Skip to first unread message

Jonathan Wood

unread,
May 28, 2009, 12:12:07 PM5/28/09
to
I'm using the following code to run a command-line program that reads a data
file:

using (Process p = new Process())
{
p.StartInfo.FileName = _exePath;
p.StartInfo.Arguments = arguments;
p.StartInfo.UseShellExecute = false;
p.StartInfo.CreateNoWindow = true;
p.StartInfo.WindowStyle = ProcessWindowStyle.Hidden;
p.StartInfo.RedirectStandardOutput = true;
p.Start();
_standardOutput = p.StandardOutput.ReadToEnd();
p.WaitForExit();
exitCode = p.ExitCode;
}

After this code runs, I then attempt to delete the data file that was read
by the program.

Normally, everything works fine. But, every once in a while, I get an
'access denied' error when I attempt to delete the data file.

How can the data file be locked after the code above has finished?

Thanks for any thoughts.

Jonathan

Peter Duniho

unread,
May 28, 2009, 12:51:57 PM5/28/09
to
On Thu, 28 May 2009 09:12:07 -0700, Jonathan Wood <jw...@softcircuits.com>
wrote:

The most likely cause is that some other process has the file locked. And
the most likely candidate is Windows Explorer. Do you have an Explorer
window open showing the file in question when your program attempts to
delete it? If so, it could be Explorer. If not, it might be an indexing
service (such as the one built into Windows).

In any case, you simply can't be sure that your process is the only
process that might be accessing the file. If it's important to get the
file deleted, you'll have to put try/catch re-try logic into a bounded
loop (so that if you haven't succeeded after some reasonable time, you do
go ahead and report the failure).

Pete

Jonathan Wood

unread,
May 28, 2009, 1:10:23 PM5/28/09
to
"Peter Duniho" <NpOeS...@nnowslpianmk.com> wrote in message
news:op.uunfs...@macbook-pro.local...

> The most likely cause is that some other process has the file locked. And
> the most likely candidate is Windows Explorer. Do you have an Explorer
> window open showing the file in question when your program attempts to
> delete it? If so, it could be Explorer. If not, it might be an indexing
> service (such as the one built into Windows).

I'm actually downloading the data file on the spot before I run the software
on it, so it's hard to imagine how anything else could have a lock on it.
The error can occur when no Windows Explorer window are open.

> In any case, you simply can't be sure that your process is the only
> process that might be accessing the file. If it's important to get the
> file deleted, you'll have to put try/catch re-try logic into a bounded
> loop (so that if you haven't succeeded after some reasonable time, you do
> go ahead and report the failure).

Right. I was just hoping to understand what is happening a little better.

Jonathan

Jonathan Wood

unread,
May 28, 2009, 1:29:19 PM5/28/09
to
In addition, although the error happens somewhat randomly, if I put a
5-second delay after this code I don't seem to get the error.

It really seems like something is keeping a lock. (Note that this code runs
in a worker thread, but I can't seem anything I'm doing with threads that
cold be a problem.)

Jonathan

"Peter Duniho" <NpOeS...@nnowslpianmk.com> wrote in message
news:op.uunfs...@macbook-pro.local...

Jeff Johnson

unread,
May 28, 2009, 2:18:42 PM5/28/09
to
"Jonathan Wood" <jw...@softcircuits.com> wrote in message
news:%23w1sxn7...@TK2MSFTNGP04.phx.gbl...

> In addition, although the error happens somewhat randomly, if I put a
> 5-second delay after this code I don't seem to get the error.
>
> It really seems like something is keeping a lock. (Note that this code
> runs in a worker thread, but I can't seem anything I'm doing with threads
> that cold be a problem.)

http://ccollomb.free.fr/unlocker/


Jonathan Wood

unread,
May 28, 2009, 7:07:24 PM5/28/09
to
I downloaded and installed that. But the lock apparently only happens for a
second and I was unable to see it with this program.

Thanks.

Jonathan

"Jeff Johnson" <i....@enough.spam> wrote in message
news:OHrz8C83...@TK2MSFTNGP03.phx.gbl...

Peter Duniho

unread,
May 28, 2009, 7:33:32 PM5/28/09
to
On Thu, 28 May 2009 10:29:19 -0700, Jonathan Wood <jw...@softcircuits.com>
wrote:

> In addition, although the error happens somewhat randomly, if I put a

> 5-second delay after this code I don't seem to get the error.
>
> It really seems like something is keeping a lock. (Note that this code
> runs in a worker thread, but I can't seem anything I'm doing with
> threads that cold be a problem.)

I'm _sure_ "something is keeping a lock". The OS doesn't typically go
around lying to you about stuff like that. :)

As I said before, I think it's likely that the file is locked from outside
your process. This assumes, of course, that you've ruled out the
possibility of a process under your control having a lock on the file, but
that seems like a safe assumption.

So, assuming you've done your homework and have ruled that out, the only
possibility left is something else. And you can't generally control the
"something else". Once a file is on the disk, the OS in particular can do
with it whatever it feels it needs to, and that might involve having the
file locked for a short period of time.

If the lock is coming from Explorer or the indexing service, it's possible
you can eliminate the problem simply by making sure there's no Explorer
window or control open (note that this includes the file dialogs), and by
disabling the indexing service. But otherwise, you will still always run
the risk of some other process noticing the file and trying to access it.
You need to handle that condition in your code.

(I have a vague recollection of a file sharing mode that allows for
deletion, but I'm pretty sure when you use that, you have to have control
over the process that's locking the file, so that you can use that mode).

Pete

Jonathan Wood

unread,
May 28, 2009, 8:04:16 PM5/28/09
to
"Peter Duniho" <NpOeS...@nnowslpianmk.com> wrote in message
news:op.uunyd...@macbook-pro.local...

> As I said before, I think it's likely that the file is locked from outside
> your process. This assumes, of course, that you've ruled out the
> possibility of a process under your control having a lock on the file, but
> that seems like a safe assumption.

Well, I'm running another process on that file, as you know, so that would
be the process in question.

However, if I don't run the program, if I just download the program and
delete it, then there is no error. Only when I run a program does the error
occasionally occur. If another process such as Windows Explorer were locking
it, I would expect that to be the case whether I ran the program or not.

Unless, of course, the program took action that caused Windows Explorer (or
whatever) to lock a file, which I don't really get.

> So, assuming you've done your homework and have ruled that out, the only
> possibility left is something else. And you can't generally control the
> "something else". Once a file is on the disk, the OS in particular can do
> with it whatever it feels it needs to, and that might involve having the
> file locked for a short period of time.
>
> If the lock is coming from Explorer or the indexing service, it's possible
> you can eliminate the problem simply by making sure there's no Explorer
> window or control open (note that this includes the file dialogs), and by
> disabling the indexing service. But otherwise, you will still always run
> the risk of some other process noticing the file and trying to access it.
> You need to handle that condition in your code.

I can download a file a million times and delete it without an error. Once I
run my other program, then I occasionally get an error. There appears to be
something left over somehow from that program.

I get your point and I can write code to handle the condition. But, at the
frequency it's happening, my thinking is that I have a bigger issue to
resolve.

Jonathan

Peter Duniho

unread,
May 28, 2009, 8:15:39 PM5/28/09
to
On Thu, 28 May 2009 17:04:16 -0700, Jonathan Wood <jw...@softcircuits.com>
wrote:

> "Peter Duniho" <NpOeS...@nnowslpianmk.com> wrote in message

> news:op.uunyd...@macbook-pro.local...
>
>> As I said before, I think it's likely that the file is locked from
>> outside your process. This assumes, of course, that you've ruled out
>> the possibility of a process under your control having a lock on the
>> file, but that seems like a safe assumption.
>
> Well, I'm running another process on that file, as you know, so that

> would be the process in question. [...]

Is this a process you have the code for? Are you sure it's closing the
file properly?

If it's not, then it's possible that you are managing to race with the OS,
as it has to clean up the open file handle for the process that just
exited (it obviously can't do that before the process exits). But
assuming you know the file is being closed properly by the process, then
by the time you see the process exit, I just don't see how the file could
still be open.

I suppose stranger things have happened though. All that said, it's
barely a .NET problem, never mind a C#-specific thing. You may get more
informed responses in a different forum.

Pete

Jonathan Wood

unread,
May 28, 2009, 11:36:58 PM5/28/09
to
"Peter Duniho" <NpOeS...@nnowslpianmk.com> wrote in message
news:op.uun0c...@macbook-pro.local...

>> Well, I'm running another process on that file, as you know, so that
>> would be the process in question. [...]
>
> Is this a process you have the code for? Are you sure it's closing the
> file properly?

Originally, it was the AVG command line anti-virus software. But after a
while, I wrote my own program. My program is a simple C++ program (not
Windows) that simply opens the file for read and write access, builds a
checksum of every byte in the file, and then closes the file. My program
checks for every possible error at every stage. I'm able to see that my
program reports no errors were encountered.

Whether I run the AVG utiltity or my own program, the file cannot be deleted
afterwards in about 1 out of 5 cases.

Note that AVG has some low-priority options that may or may not spawn
additional threads or processes. That's why I wanted to rule that out, even
though the return value from the AVG software appeared valid.

> If it's not, then it's possible that you are managing to race with the OS,
> as it has to clean up the open file handle for the process that just
> exited (it obviously can't do that before the process exits). But
> assuming you know the file is being closed properly by the process, then
> by the time you see the process exit, I just don't see how the file could
> still be open.

That makes two of us--I'm really at a loss. I supposed there is a
possibility that it takes a moment to REALLY close a file and my program is
returning before Windows can clean everything up. But that's a guess at
best.

> I suppose stranger things have happened though. All that said, it's
> barely a .NET problem, never mind a C#-specific thing. You may get more
> informed responses in a different forum.

Well, I'm new to using Process this way. This is the object that tells me
when the program is done. I may be able to get more help elsewhere but I
just don't have enough information to know where or to post meaningful
information to anyone not using Process.

Thanks.

Jonathan

Peter Duniho

unread,
May 29, 2009, 12:13:43 AM5/29/09
to
On Thu, 28 May 2009 20:36:58 -0700, Jonathan Wood <jw...@softcircuits.com>
wrote:

> [...]


> Well, I'm new to using Process this way. This is the object that tells
> me when the program is done. I may be able to get more help elsewhere
> but I just don't have enough information to know where or to post
> meaningful information to anyone not using Process.

If you really want to preserve that exact context, you can look at the
unmanaged Win32 CreateProcess() function, as well as OpenProcess() and
WaitForSingleObject() (to get the process handle, and to wait for process
exit, respectively). Of course, that assumes you're comfortable writing a
regular unmanaged Win32 application.

There are regular ".win32." programming newsgroups; if you think that the
only process locking the file is the one you create, and you think someone
else might be able to offer a better suggestion than just retrying over
some short period of time, you should try one of those. You may in fact
be able to ask the question more generally (e.g. "I'm waiting for the
process to exit, but even after that happens, it still seems to have the
file locked briefly"), but of course as always they may find the question
more answerable if you provide a concise-but-complete code example (which
I'll note you haven't even provided here :) ).

If I have time, I may try to code a short sample to try to reproduce the
problem. But, it seems to me that if you really want help, you should be
the one doing that. :)

Pete

Jonathan Wood

unread,
May 29, 2009, 2:13:54 AM5/29/09
to
"Peter Duniho" <NpOeS...@nnowslpianmk.com> wrote in message
news:op.uuobc...@macbook-pro.local...

> If you really want to preserve that exact context, you can look at the
> unmanaged Win32 CreateProcess() function, as well as OpenProcess() and
> WaitForSingleObject() (to get the process handle, and to wait for process
> exit, respectively). Of course, that assumes you're comfortable writing a
> regular unmanaged Win32 application.

Yes, I could do that. But my gut feeling is that it won't duplicate my
current scenario exactly and won't repro the problem, and the result would
be no help at all. There's a lot more going on under a .NET app and it seems
related to .NET somehow.

At any rate, maybe I'll do that if I can't find anything else. But it really
seems like a long shot to me.

> There are regular ".win32." programming newsgroups; if you think that the
> only process locking the file is the one you create, and you think someone
> else might be able to offer a better suggestion than just retrying over
> some short period of time, you should try one of those.

Of course, if I mention the actual code, they'll all tell me I'm in the
wrong group. :-)

> You may in fact be able to ask the question more generally (e.g. "I'm
> waiting for the process to exit, but even after that happens, it still
> seems to have the file locked briefly"), but of course as always they may
> find the question more answerable if you provide a concise-but-complete
> code example (which I'll note you haven't even provided here :) ).

It's a large project. If I thought it'd be helpful at all, I'd be happy to
provide more code than I've posted already.

Anyway, perhaps I will do something more along the lines of what you're
talking about if I don't find a solution. I've already let some important
work slip because of this and so it'll have to wait a little if I need to
write more test code--especially if I consider that code to be a long shot.

Thanks.

Jonathan

Peter Duniho

unread,
May 29, 2009, 2:31:41 AM5/29/09
to
On Thu, 28 May 2009 23:13:54 -0700, Jonathan Wood <jw...@softcircuits.com>
wrote:

> "Peter Duniho" <NpOeS...@nnowslpianmk.com> wrote in message

> news:op.uuobc...@macbook-pro.local...
>
>> If you really want to preserve that exact context, you can look at the
>> unmanaged Win32 CreateProcess() function, as well as OpenProcess() and
>> WaitForSingleObject() (to get the process handle, and to wait for
>> process exit, respectively). Of course, that assumes you're
>> comfortable writing a regular unmanaged Win32 application.
>
> Yes, I could do that. But my gut feeling is that it won't duplicate my
> current scenario exactly and won't repro the problem, and the result
> would be no help at all. There's a lot more going on under a .NET app
> and it seems related to .NET somehow.

Well, then at least reproduce it using .NET.

> [...]


>> You may in fact be able to ask the question more generally (e.g. "I'm
>> waiting for the process to exit, but even after that happens, it still
>> seems to have the file locked briefly"), but of course as always they
>> may find the question more answerable if you provide a
>> concise-but-complete code example (which I'll note you haven't even
>> provided here :) ).
>
> It's a large project. If I thought it'd be helpful at all, I'd be happy
> to provide more code than I've posted already.

I went ahead and made an attempt. However, I'm unable to reproduce the
problem you're reporting. See below for the code I wrote. It's been
running for the last few minutes, and is up to about 3000 iterations
without any errors.

The fact that it's a large project is of no real consequence. You
shouldn't be posting the whole project anyway. You should be narrowing
down the problem until you have the smallest amount of code that still
reproduces the problem. And frankly, at this point, I really don't see
any way forward, at least not for me. It's not really possible for anyone
else to offer reliable advice without being able to reproduce the problem
themselves.

And of course, you have a perfectly reasonable work-around. If there's a
bug in your code causing the problem, it would be much better to find and
fix that instead. But if you're short on time, the work-around is easy,
and you can revisit the problem later.

Now up to 6000 iterations, by the way. :) Still no errors.

Pete


Main program:

using System;
using System.IO;
using System.Diagnostics;

namespace TestProcessFileLock
{
class Program
{
static void Main(string[] args)
{
try
{
Random rnd = new Random();
int iter = 0;

while (true)
{
string strFilename = Path.GetTempFileName();

using (FileStream fs = new FileStream(strFilename,
FileMode.Truncate))
{
byte[] rgb = new byte[1024];
int cb = rnd.Next(100 * 1024, 500 * 1024);

while (cb > 0)
{
int cbWrite = Math.Min(rgb.Length, cb);

rnd.NextBytes(rgb);
fs.Write(rgb, 0, cbWrite);
cb -= cbWrite;
}
}

Process process = new Process();

process.StartInfo.FileName = "DummyProcess.exe";
process.StartInfo.Arguments = "\"" + strFilename +
"\"";
process.StartInfo.UseShellExecute = false;
process.StartInfo.CreateNoWindow = true;
process.StartInfo.RedirectStandardOutput = true;

Console.WriteLine("{0}: \"{1}\"", (++iter).ToString(),
strFilename);
process.Start();

string strOutput;

while ((strOutput = process.StandardOutput.ReadLine())
!= null)
{
Console.WriteLine("output: \"{0}\"", strOutput);
}
process.WaitForExit();
if (process.ExitCode != 0)
{
Console.WriteLine("process exited with error");
Console.ReadLine();
}

File.Delete(strFilename);
}
}
catch (Exception exc)
{
Console.WriteLine("Error: \"" + exc.Message + "\"");
}

Console.ReadLine();
}
}
}


DummyProgram.exe (make sure a copy of this .exe is in the same directory
as the main program):

using System;
using System.IO;

namespace DummyProcess
{
class Program
{
static int Main(string[] args)
{
int ret = 0;

if (args.Length < 1)
{
Console.WriteLine("must provide a filename as the first
command-line argument");
ret = 1;
}

try
{
using (FileStream fs = new FileStream(args[0],
FileMode.Open))
{
byte[] rgb = new byte[1024];

Console.WriteLine("reading {0} bytes",
fs.Length.ToString());

while (fs.Read(rgb, 0, rgb.Length) > 0)
{
}
}
}
catch (Exception exc)
{
Console.WriteLine("Error: \"" + exc.Message + "\"");
ret = 1;
}

return ret;
}
}
}

Nick

unread,
May 29, 2009, 3:47:10 AM5/29/09
to
Have you tried running Process Monitor from Sysinternals.

http://technet.microsoft.com/en-us/sysinternals/bb896645.aspx

Only monitor File System Activity.
Then set up filter on path ending in the filename.

Run your program.

This should give you a dump of events related to the file.

Jonathan Wood

unread,
May 29, 2009, 11:02:47 AM5/29/09
to
I'll check this out. Thanks.

Jonathan

"Nick" <Ni...@spam.com> wrote in message
news:001b0d18$0$2058$c3e...@news.astraweb.com...

Jonathan Wood

unread,
May 29, 2009, 10:57:48 AM5/29/09
to
Thanks for spending time on this.

I can't do it now but perhaps I'll try and create a sample that shows the
problem. It could take a while because there are a number of variables that
may or may not be related. I did post in a Win32 group as you suggested and
one person offered the following:

> According to my experiences when the process handle becomes signaled
> (terminates) it's likely (especially on multi-processor/core systems) that
> file handles are still open. This may be the side effect of the fact that
> handles live in kernel-mode and the cleanup is done after the process
> handle is signaled.
>
> You either can either use FILE_FLAG_DELETE_ON_CLOSE and share the handle
> or if you can't use that you should sleep for example 200 or 500 ms and
> try to delete again. You also should do this when this specific error code
> or exception appears because other error codes most likely indicate error
> conditions that cannot be solved by a loop.

Also note that sleeping as little as 500ms before deleting the file appears
to completely eliminate the error. So I need to think about the best way to
stay productive on this project.

Thanks, I'll post back if there are any developments.

Jonathan

"Peter Duniho" <NpOeS...@nnowslpianmk.com> wrote in message

news:op.uuohq...@macbook-pro.local...

florin

unread,
Jun 9, 2009, 3:34:01 AM6/9/09
to
Or you can just run the "Handle" tool from Sysinternals (now bought by
Microsoft) which can show you who is having handles to a given file.
You can just run "handle.exe <my file name> >someLogFile.txt" before
you do the delete, in case the delete fails just look in the
someLogFile.txt to see who had that file open at that time. Once you
know who is to blame you will know how to fix/hack it.

florin

0 new messages