subprocess failure (Windows-specific ?)

[Bertrand Augereau]

Hello everybody,

I'm trying to drive a WIN32 application from a Racket gui app to provide the QA guys a simple way to interact with it. So far I'm quite successful but I'm having a little issue.

I spawn my subprocess with subprocess : https://docs.racket-lang.org/reference/subprocess.html

And for the sake of completeness I want to know when suprocess failed (because the users tampered with the exe, because there's an ACL issue, whatever).

The function doesn't have a return value for failure and it doesn't raise an exception when I do stuff like

(subprocess "no exe here")

or

(subprocess "nice picture of an aardvark.png")

It only returns a seemingly valid value.

Can someone help me ? Is it an API defect and a special value should be returned ? Should an exception be raised ? The doc doesn't say much about this.

I don't really care about the OS-level cause of the failure but I need to know if it failed.

I didn't test on my Linux box for consistency.

By the way, subprocess-pid returns 0 on such a "never spawned" process, and I think it should be an error case too.

Cheers,

Bertrand

PS: Here's the reddit thread where I asked how to do it:

https://www.reddit.com/r/Racket/comments/f0cbrp/is_there_a_way_to_catch_subprocess_failures/

[George Neuner]

On 2/8/2020 3:37 AM, Bertrand Augereau wrote:

  :

I spawn my subprocess with subprocess : https://docs.racket-lang.org/reference/subprocess.html

And for the sake of completeness I want to know when suprocess failed (because the users tampered with the exe, because there's an ACL issue, whatever).

The function doesn't have a return value for failure and it doesn't raise an exception when I do stuff like

(subprocess "no exe here")

or

(subprocess "nice picture of an aardvark.png")

It only returns a seemingly valid value.

The 1st value returned by (subprocess) is an opaque reference to the executing process.  If you pass the reference to (subprocess-status) it will return  'running  if the process currently is executing, or the exit/error value.

Can someone help me ? Is it an API defect and a special value should be returned ? Should an exception be raised ? The doc doesn't say much about this.

A special value is returned - the opaque reference.  Perhaps the docs could use a simple example of how to use the functions effectively.  Try playing a bit with the following:

#lang racket

(define sub #f)

(let [
      (name "C:\\WINDOWS\\system32\\notepad.exe")
      (in #f)(out #f)(err #f)
     ]
  (set!-values (sub out in err) (subprocess #f #f #f name))
  )

(subprocess-status sub)
(subprocess-pid sub)

(subprocess-wait sub)
;; close subprogram here

(subprocess-status sub)
(subprocess-pid sub)

I don't really care about the OS-level cause of the failure but I need to know if it failed.

I didn't test on my Linux box for consistency.

By the way, subprocess-pid returns 0 on such a "never spawned" process, and I think it should be an error case too.

No actual process can have pid 0 ... that value is reserved  (on Unix/Linux and Windows). 

Racket doesn't support fork (because Windows doesn't), but recall that when you fork in Unix/Linux, the child process receives the illegal pid value 0 so that it knows it is the child, while the parent receives the real pid value for the child.

Also note that (subprocess-pid) will be valid only if the process started successfully.


Hope this helps,
George

[Bertrand Augereau]

The 1st value returned by (subprocess) is an opaque reference to the executing process.  If you pass the reference to (subprocess-status) it will return  'running  if the process currently is executing, or the exit/error value.

"exit/error value" is the issue there.
(subprocess-status ps) will return 'running if the process is running.
If it doesn't, either the process has never ran, either it has finished with a exit code.
No way (that I see) to disambiguate.

 

No actual process can have pid 0 ... that value is reserved  (on Unix/Linux and Windows). 

Racket doesn't support fork (because Windows doesn't), but recall that when you fork in Unix/Linux, the child process receives the illegal pid value 0 so that it knows it is the child, while the parent receives the real pid value for the child.

Also note that (subprocess-pid) will be valid only if the process started successfully.

 

(subprocess-pid) is only valid when (subprocess-status) is 'running but testing the status then requesting the pid is not atomic, therefore subprocess-pid should return #f or raise. Or return pid 0 but it should be documented :)
By the way there is a process with pid 0 on Linux, it's "sched". And on Windows, its "idle". Of course it could never be the result of (subprocess ...) but I don't want to rely on incidental behaviour from an API :=

Hope this helps,
George

Thanks for contributing, it's much appreciated,

Bertrand

[George Neuner]

On 2/8/2020 8:58 AM, Bertrand Augereau wrote:

The 1st value returned by (subprocess) is an opaque reference to the executing process.  If you pass the reference to (subprocess-status) it will return  'running  if the process currently is executing, or the exit/error value.

"exit/error value" is the issue there.
(subprocess-status ps) will return 'running if the process is running.
If it doesn't, either the process has never ran, either it has finished with a exit code.
No way (that I see) to disambiguate.

I'm not sure I completely understand the problem.  You're correct that there's no way to tell whether the value is an exit code from the program or an error from the operating system ... but there also is no way to tell that starting the program from the shell  IF  you rely solely on the exit code.

But if the idea is to tell whether the program started correctly - even if it since has ended - then something like:

(or
  (eq? 'running (subprocess-status ps))
  (not (= 0 (subprocess-pid ps)))
  )

should do the trick.

(subprocess-pid) is only valid when (subprocess-status) is 'running but testing the status then requesting the pid is not atomic, therefore subprocess-pid should return #f or raise. Or return pid 0 but it should be documented :)

It does return 0 if the program never started.

I won't argue about documentation, but I will note that the Racket "process" object retains the assigned pid value even after the process terminates ... so testing whether it is running before retrieving the pid does not seem to be a real burden.  Do you have a use case where this matters?

By the way there is a process with pid 0 on Linux, it's "sched". And on Windows, its "idle". Of course it could never be the result of (subprocess ...) but I don't want to rely on incidental behaviour from an API :=

Those are not processes - they are kernel functions called when no actual process is ready to run.  They consume time and CPU cycles, so they are included in the process listing, but they really are just system overhead.  When the OS is in one of these functions it is sleeping waiting for either a timer tick or an I/O completion.

And pid 0 will never be assigned to a real process.

Hope this helps,
George

[Bertrand Augereau]

I'm not sure I completely understand the problem.  You're correct that there's no way to tell whether the value is an exit code from the program or an error from the operating system ... but there also is no way to tell that starting the program from the shell  IF  you rely solely on the exit code.

That's why CreateProcess and fork/exec return code.

If I was coding this in C, I would have no issue, so I think the problem should be clearly addressed in Racket too.

I don't see what the shell's got to do with it :)

 

But if the idea is to tell whether the program started correctly - even if it since has ended - then something like:

(or
  (eq? 'running (subprocess-status ps))
  (not (= 0 (subprocess-pid ps)))
  )

should do the trick.

Yes in practice, it might be good enough. 

(subprocess-pid) is only valid when (subprocess-status) is 'running but testing the status then requesting the pid is not atomic, therefore subprocess-pid should return #f or raise. Or return pid 0 but it should be documented :)

It does return 0 if the program never started.

I won't argue about documentation, but I will note that the Racket "process" object retains the assigned pid value even after the process terminates ... so testing whether it is running before retrieving the pid does not seem to be a real burden.  Do you have a use case where this matters?

You're using undocumented behaviour there, Plausible one though.

But it's subject to change in the future so I'd prefer to address it and have the documentation changed by the Racket team so that my code is robust in the long run :)

By the way there is a process with pid 0 on Linux, it's "sched". And on Windows, its "idle". Of course it could never be the result of (subprocess ...) but I don't want to rely on incidental behaviour from an API :=

Those are not processes - they are kernel functions called when no actual process is ready to run.  They consume time and CPU cycles, so they are included in the process listing, but they really are just system overhead.  When the OS is in one of these functions it is sleeping waiting for either a timer tick or an I/O completion.

And pid 0 will never be assigned to a real process.

Indeed, but if 0 is a magic number of sorts for subprocess-pid, it should be documented too.

IMO, it would be cleaner to have subprocess returns #f or raise an exception if spawning the process fails. It would solve my initial problem and mimic what the C APIs do.

The subprocess-pid would not need any further consideration then :)

[Matthew Flatt]

At Sat, 8 Feb 2020 17:08:18 +0100, Bertrand Augereau wrote:
> > I'm not sure I completely understand the problem. You're correct that
> > there's no way to tell whether the value is an exit code from the program
> > or an error from the operating system ... but there also is no way to tell
> > that starting the program from the shell IF you rely solely on the exit
> > code.
> >
>
> That's why CreateProcess and fork/exec return code.

Currently, if fork() fails on Unix (e.g., because there are too many
processes), then `subprocess` will raise an exception. But if fork()
succeeds, then there's normally no way to communicate an error from
exec() except through the exit code, since exec() is in the child
process.[*] So, that's why there's no way to distinguish "program
couldn't start" from "program exited with a non-0 status" on Unix.

You're right that `subprocess` could make a distinction on Windows with
CreateProcess(). Currently, `subprocess` doesn't make a distinction,
mostly because not doing so makes the behavior somewhat more consistent
with conventional Unix behavior.

[*] It seems possible to set up an extra channel of communication to
report whether exec() succeeds. I think that would be tricky at
best, though, because it's not clear how to clean up the channel if
exec() does succeed.

> > But if the idea is to tell whether the program started correctly - even if
> > it since has ended - then something like:
> >
> > (or
> > (eq? 'running (subprocess-status ps))
> > (not (= 0 (subprocess-pid ps)))
> > )
> >
> > should do the trick.
> >
>
> Yes in practice, it might be good enough.

I agree that this is not a good choice.

For compatibility while also exposing the result of CreateProcess() on
Windows, a new `subprocess-...` operation is probably best. I'll look
into adding that.

[Bertrand Augereau]

Hi Matthew,

Currently, if fork() fails on Unix (e.g., because there are too many
processes), then `subprocess` will raise an exception. But if fork()
succeeds, then there's normally no way to communicate an error from
exec() except through the exit code, since exec() is in the child
process.[*] So, that's why there's no way to distinguish "program
couldn't start" from "program exited with a non-0 status" on Unix.

You're right that `subprocess` could make a distinction on Windows with
CreateProcess(). Currently, `subprocess` doesn't make a distinction,
mostly because not doing so makes the behavior somewhat more consistent
with conventional Unix behavior.

You're right, but wouldn't using the posix_spawn family have better semantics, better performance, and would allow to unify between POSIX and Windows behaviours nicely ? :)

Cheers,

Bertrand

[Matthew Flatt]

At Sat, 8 Feb 2020 17:46:06 +0100, Bertrand Augereau wrote:
> You're right, but wouldn't using the posix_spawn family have better
> semantics, better performance, and would allow to unify between POSIX and
> Windows behaviours nicely ? :)

It's the usual problem: posix_spawn() doesn't quite support all of the
things Racket does between fork() and exec().

[Bertrand Augereau]

Of course !

Inter-OS APIs are such a pain. I think you do the right thing by explicitating the differences regarding various OSes on the scribbled doc page.

Maybe just documenting that:

* subprocess-pid retains the pid forever after the child has stopped running

* subprocess-pid returns 0 (or invalid-pid) on Windows and BSD and Linux and OSX (who knows the bright future of Racket ? :) ) if the process spawning failed

would be sufficient to write production code with defined behaviour. I'd certainly be satisfied.

Cheers,

Bertrand

--
You received this message because you are subscribed to the Google Groups "Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to racket-users...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/racket-users/5e3ee899.1c69fb81.a3dd4.306bSMTPIN_ADDED_MISSING%40gmr-mx.google.com.