Subprocess .wait() is not waiting

[adam.c.bernier]

Hi,

I am on Windows 7. Python 2.7

I'm trying to have a program run another program using `subprocess.Popen`

import subprocess as sp

args = shlex.split(args)
proc = sp.Popen(args,stdout=sp.PIPE,stderr=sp.PIPE)
out, err = proc.communicate()
proc.wait()

But it *sometimes* doesn't wait and the other program -- which generates a set of 14 Excel files -- does not complete before control is returned to the calling program.

Any ideas on what I can do to make `subprocess.wait()` actually wait?

Thanks in advance for any assistance you might be able to provide.

[Ian Kelly]

If that's what's happening it would be a bug. Are you sure that the
other program isn't simply crashing or otherwise failing to complete?

By the way, the wait in the code above is redundant because
communicate already waits.

[Chris Angelico]

On Sat, Mar 25, 2017 at 6:40 AM, Ian Kelly <ian.g...@gmail.com> wrote:
> If that's what's happening it would be a bug. Are you sure that the
> other program isn't simply crashing or otherwise failing to complete?
>

Or possibly is running asynchronously. Is this a GUI app? A lot of
Windows GUI programs don't wait when you invoke them.

ChrisA

[adam.c.bernier]

Aha! Since the Excel app is a GUI app it's not waiting is that right?

[adam.c.bernier]

If that's the case is there any workaround?

[adam.c.bernier]

Thank you for the comment about the wait being redundant. Much appreciated.

[Chris Angelico]

Errr.... been a while since I messed with Windows.... from memory, I
think you can "start /wait programname" to make it wait?? Worth a try,
at least.

Otherwise, your best bet would be to wait in a completely different
way. Give the app a specific file name to use as a signal, and when
it's done, it should create that file. You then watch for that file,
and when it exists, you move on.

ChrisA

[Ian Kelly]

On Fri, Mar 24, 2017 at 2:09 PM, Chris Angelico <ros...@gmail.com> wrote:

> Errr.... been a while since I messed with Windows.... from memory, I
> think you can "start /wait programname" to make it wait?? Worth a try,
> at least.

start /wait is for batch scripts, specifically because by default
start doesn't wait on GUI programs. I'm pretty sure that
WaitForSingleObject (which Popen.wait uses) is perfectly capable of
waiting on GUI programs (I would even bet dollars for donuts that this
is exactly what start /wait does).

Another possibility is that the subprocess itself is creating its own
subprocesses and not waiting on them. If it terminates before its
children, the wait() call will finish even though work is still being
done.

[adam.c.bernier]

Thank you! I will try the first answer here http://stackoverflow.com/questions/11615455/python-start-new-command-prompt-on-windows-and-wait-for-it-finish-exit and see where that gets me. Thanks again for your thoughts.

[eryk sun]

On Fri, Mar 24, 2017 at 6:42 PM, adam.c.bernier <adam.c....@kp.org> wrote:
>
> I am on Windows 7. Python 2.7
>
> I'm trying to have a program run another program using `subprocess.Popen`
>
> import subprocess as sp
>
> args = shlex.split(args)

Is this command supposed to run cross-platform? If not, then splitting
it into a list is pointless. Windows uses a comand-line string, and
Popen will just have to rebuild the command line from the list.

> proc = sp.Popen(args,stdout=sp.PIPE,stderr=sp.PIPE)
> out, err = proc.communicate()
> proc.wait()
>
> But it *sometimes* doesn't wait and the other program -- which generates a set of
> 14 Excel files -- does not complete before control is returned to the calling program.
>
> Any ideas on what I can do to make `subprocess.wait()` actually wait?
>
> Thanks in advance for any assistance you might be able to provide.

Without knowing the command you're running, all we can do is
speculate. It could be that it's an application that uses a single
instance, in which case running another instance simply messages the
main process and then exits.

[adam.c.bernier]

Thank you, eryk sun. The command is calling a Python script which makes calls to openpyxl to generate 14 Excel files.

[eryk sun]

On Fri, Mar 24, 2017 at 8:24 PM, Ian Kelly <ian.g...@gmail.com> wrote:
> On Fri, Mar 24, 2017 at 2:09 PM, Chris Angelico <ros...@gmail.com> wrote:
>

>> Errr.... been a while since I messed with Windows.... from memory, I
>> think you can "start /wait programname" to make it wait?? Worth a try,
>> at least.
>

> start /wait is for batch scripts, specifically because by default
> start doesn't wait on GUI programs. I'm pretty sure that

In interactive mode, cmd waits on console programs but not GUI
programs. In a batch script or /c or /k command it always waits, in
which case `start` is required to run without waiting. The /w[ait]
option may be needed if you're using `start` for some other feature,
such as creating a new console or setting the CPU affinity. Otherwise
if you just need to wait in a batch script, then you probably don't
need `start /w`.

[eryk sun]

openpyxl should do all of the work in the process that Popen is
waiting on. Is the command-line in `args` something like
"C:\Python27\python.exe path\to\script.py"?

[adam.c.bernier]

That's exactly right.