Deadlock when logging to stdout and stderr
Romuald
I created for my company a build and tests system which is mainly based on
PS scripts. As we use it to build different langages (Java, C#, C++), my
system uses external program to effectively run the builds and the tests.
To run external software, I use a ProcessStartInfo object, like this:
$si = New-Object -TypeName System.Diagnostics.ProcessStartInfo
$si.FileName = "cvs.exe"
$si.RedirectStandardOutput = $true
$si.RedirectStandardError = $true
$si.UseShellExecute = $false
$si.RedirectStandardOutput = $true
$si.RedirectStandardError = $true
$p = [diagnostics.process]::Start($si)
$output = $p.StandardOutput.ReadToEnd()
$errors = $p.StandardError.ReadToEnd()
$p.WaitForExit()
Log-Message $log Info $output
if($errors.Length -gt 0)
{
Log-Message $log ERROR $errors
}
I'm facing a pretty blocking issue with thr stderr and stdout redirection,
in some case, when the amount of traces is pretty big, my PS script is
blocked completely blocked. I checked the external program is well
terminated, so the problem doesn't seem to come from external programs (which
are pretty different) but from Powershell which doesn't handle redirection
correctly.
Or the error is on my side.
I tried to remove the redirection and everything works well, if I ask the
developer to stop their output traces during the test, same thing, everything
works well.
So, I'm here to find a little help.
Thanks in advance.
Alex K. Angelopoulos dot dot again at
error buffer fills up while you're waiting for the other buffer, producing a
deadlock. See here for a brief description of the problem:
The standard solution is to handle reading output and reading error from
distinct threads - something that I'm not well-acquainted with, but that
_can_ be done. Since this has been sitting here a week with no responses,
ping back if you're still trying to resolve this and I can do some more
looking. This problem definitely _can_ be solved.
"Romuald" <Rom...@discussions.microsoft.com> wrote in message
news:A7E99F3E-BC07-4152...@microsoft.com...
Romuald
This seems to be exactly the problem. So, now, I just have to implement this
solution.
Many thanks again, your link is really helpful.
Alex K. Angelopoulos dot dot again at
Which version of PowerShell are you using? PS1 or PS2?
"Romuald" <Rom...@discussions.microsoft.com> wrote in message
news:DACDF798-C329-451C...@microsoft.com...
Ewok
OldDog
> I am using v2 and would like to asyn tail many logs...at the same time
>
>
>
> "Alex K. Angelopoulos" wrote:
> > Which version of PowerShell are you using? PS1 or PS2?
>
> >news:DACDF798-C329-451C...@microsoft.com...
> > > Many thanks Alex,
> > > This seems to be exactly the problem. So, now, I just have to implement
> > > this
> > > solution.
>
> > > Many thanks again, your link is really helpful.
>
> > > "Alex K. Angelopoulos" wrote:
>
> > >> I'm fairly certain that your problem here is that either your output or
> > >> your
> > >> error buffer fills up while you're waiting for the other buffer,
> > >> producing a
> > >> deadlock. See here for a brief description of the problem:
>
>
> > >> The standard solution is to handle reading output and reading error from
> > >> distinct threads - something that I'm not well-acquainted with, but that
> > >> _can_ be done. Since this has been sitting here a week with no responses,
> > >> ping back if you're still trying to resolve this and I can do some more
> > >> looking. This problem definitely _can_ be solved.
>
>
> - Show quoted text -
Big dummy here, but I see in the script that
$si.RedirectStandardOutput = $true
$si.RedirectStandardError = $true
is assigned twice. Is there a reason for that? AND using this method,
could you tell when cvs.exe starts and stops?
Alex K. Angelopoulos
news:37f996b7-73a0-497e...@d32g2000yqh.googlegroups.com...
>
> Big dummy here, but I see in the script that
>
> $si.RedirectStandardOutput = $true
> $si.RedirectStandardError = $true
>
> is assigned twice. Is there a reason for that? AND using this method,
> could you tell when cvs.exe starts and stops?
>
Look again, carefully. The first is Output, and the second is Error.
As for telling when it starts and stops, when you invoke the Start method
> $p = [diagnostics.process]::Start($si)
The process starts immediately. Telling when it stops is possible _only_ if
a buffer doesn't fill up while the worker thread is waiting on the other
line, because console applications like this will halt while waiting for an
output or an error buffer to be cleared. Each buffer is only 32k in size.
A factory analogy might make this a bit easier to grasp. Picture the csv
command as a quality control station on a conveyor-belt assembly line at the
Sartre Widget Factory. Items that meet quality standards (output) get
directed to a shipping line; ones that don't get routed to a rework line
(error).
Pushing the analogy farther, suppose you have a single worker servicing both
the shipping and rework lines. The worker has to run to the shipping line,
clear it off, and then run over to the rework line and clear it off. The job
is then done, and the worker can leave.
Unfortunately, if there is more stuff going to either line than will fit on
it, the line will shut down. Since the worker isn't smart enough to switch
lines at will and since he has to wait until all of the items are off the
line, there are all sorts of ways he can get stuck. He might be at the
shipping line while the rework line fills up; in that case, the assembly
line halts and he's stuck at the wrong line, waiting for the last item to be
put onto the shipping line. If the shipping line fills up, he may remove
everything - but then he's off to the rework line and more items may come
down the output line after he's gone. Since he can't leave until the entire
line is clear, he's stuck forever with no exit.
I was going to work in Godot as the floor supervisor, but I figured enough
was enough. ;)