Problem with popen on windows

[Kroeger Simon (ext)]

Hi all,

i'm using ruby 1.8.2 (2004-12-25) [i386-mswin32] on win2000.

When starting my GUI app with rubyw (because i don't like the empty
window) there is another window poping up whenever i call popen. (or use
backquotes to start another process)

Using just ruby.exe all seems fine (except the initial nasty empty
window)

Someone knows a way around?

cheers

Simon

[Robert Klemme]

Untested: maybe you can change COMSPEC env var in your ruby script to
point to another binary that won't open a window before executing the new
process. Another option might be to add a command line param to COMSPEC -
although I don't know whether that would work.

Kind regards

robert

[Kroeger Simon (ext)]

Hmm, this has no effect on the backqoutes thingie. It has an effect on
popen but whatever I put in COMSPEC it just stops working.

C:\WINNT\system32\cmd.exe is the standard, i tried:

C:\WINNT\system32\command.com
C:\WINNT\system32\cmd.exe /MIN
and empty

all have the same effect: no window poping up (fine) but no process is
executed also (not so fine)

why is cmd.exe started anyway?

cheers

Simon

[Alex Fenton]

I too would dearly love to find a solution to this. I have seen this
question
batted about several times over the years, but no solution.

I don't know cmd.exe is invoked, but presume it acts as the shell for
system()
``() and popen(). The relevant lines in win32/win32.c in the Ruby source
seem to be in CreateChild (caution: IANA C-programmer)

else if ((shell = getenv("COMSPEC")) &&
((redir < 0 ? has_redirection(cmd) : redir) ||
isInternalCmd(cmd, shell))) {
char *tmp = ALLOCA_N(char, strlen(shell) + len + sizeof(" /c "));
sprintf(tmp, "%s /c %.*s", shell, len, cmd);
cmd = tmp;
}

Looking at this code and the cmd.exe reference at
www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/cmd.mspx

it seemed worth trying
ENV['COMSPEC'] += ' /q'
in Ruby to see if switching cmd.exe's echo off would suppress the annoying
box pop-up. But having done this, any system, ``, popen call then fails
with ENOENT - as if Windows can't then locate the shell executable.

Using command.com works for me, but still shows a pop-up box, and I
can't see any suitable options to suppress it.

I'm stumped from here, but perh someone with a better knowledge of
Windows process internals and/or Ruby internals can help.

alex

[stevetuckner]

Kroeger Simon (ext) wrote:

Below was my attempt to say, here is how you should fix this issue. And
then it didn't work. If I run it with ruby it works fine. When I run it
with rubyw, it just goes away when I click run. Any ideas anyone? The
system code and `` code is taken from the list a long time ago.

Steve Tuckner

---------------------------------------------
require "vr/vruby"
require "vr/vrcontrol"
require 'Win32API'

def system(command)
Win32API.new("crtdll", "system", ['P'], 'L').Call(command)
end

def `(command)
popen = Win32API.new("crtdll", "_popen", ['P','P'], 'L')
pclose = Win32API.new("crtdll", "_pclose", ['L'], 'L')
fread = Win32API.new("crtdll", "fread", ['P','L','L','L'], 'L')
feof = Win32API.new("crtdll", "feof", ['L'], 'L')
saved_stdout = $stdout.clone
psBuffer = " " * 128
rBuffer = ""
f = popen.Call(command,"r")
while feof.Call( f )==0
l = fread.Call( psBuffer,1,128,f )
rBuffer += psBuffer[0..l]
end
pclose.Call f
$stdout.reopen(saved_stdout)
rBuffer
end

class MyForm < VRForm

def construct
addControl VRButton, "run", "run", 0, 0, 150, 40
move 0, 0, 160, 73
end

def run_clicked
messageBox(`dir`)
end
end

VRLocalScreen.showForm MyForm
VRLocalScreen.messageloop

[x1]

I second this and have battled it numerous times. Finally, I've just given
up on it and execute the job with ruby.exe, and let a dos prompt sit there
ignorantly.

I've been sitting here hopelessly PRAYING that a fix for this will be
provided with ruby2.

*waits impatiently.

[x1]

I just found a fix that works for me.. I hope this helps someone else out..

Nonetheless, it does require a small vb script, which acts as a
parent for the child script.. to execute, the command is as follows:

[code]
WScript RunHide.vbs Cmd /C MyBatch.cmd
[/code]

The internal source to RunHide.vbs is as follows:
##############################################################################
[code]
' ___________________________________________________________________
'
' VBScript File: RunHide.vbs
' Author: Frank-Peter Schultze
'
' Updates: http://www.fpschultze.de/w5.htm
' Enhancement Req.
' and Bug Reports: sup...@fpschultze.de
'
' Built/Tested On: Windows 2000 SP2
' Requirements: OS: Windows NT4+
' WSH: 1.0+
'
' Purpose: Create a new process that executes in a hidden
' window.
'
' Syntax: RunHide Command [Arg1 ["Arg 2" [...
'
' State Changes:
'
' Assumptions And
' Limitations:
'
' Last Update: 2004-01-29
' ___________________________________________________________________
'

Option Explicit

On Error Resume Next

Dim oSh, sCmd, i

If Not IsWscript() Then
WScript.Echo "Please run this script using WSCRIPT."
WScript.Quit(1)
End If

If ParseCmdLine(sCmd) Then
Set oSh = CreateObject("WScript.Shell")
oSh.Run Trim(sCmd), 0, False
End If

WScript.Quit(0)

Private Function ParseCmdLine(sCmd)

Dim i

ParseCmdLine = False

If WScript.Arguments.Count > 0 Then
Select Case WScript.Arguments(0)
Case "/?" : ShowUsage(True)
Case "-?" : ShowUsage(True)
Case "/h" : ShowUsage(True)
Case "-h" : ShowUsage(True)
Case "h" : ShowUsage(True)
Case Else : For i = 0 to WScript.Arguments.Count -1
If InStr(WScript.Arguments(i), " ") Then
sCmd = sCmd & " " & Chr(34) &
WScript.Arguments(i) & Chr(34)
Else
sCmd = sCmd & " " & WScript.Arguments(i)
End If
Next
ParseCmdLine = True
End Select
Else
ShowUsage(True)
End If

ParseCmdLine = True

End Function

Private Function IsWscript()

IsWscript = False

If InStrRev(LCase(WScript.FullName), "wscript.exe", -1) Then
IsWscript = True
End If

End Function

Private Sub ShowUsage(bExit)

Dim strHelp

strHelp = "Creates a new process that executes in a hidden window." & _
vbCrLf & vbCrLf & _
"RunHide Command [Arg1 [" & Chr(34) & "Arg 2" & Chr(34) & " [.."
MsgBox strHelp, 64, "RunHide Syntax"

If bExit Then
WScript.Quit(1)
End If

End Sub

[/code]
##############################################################################

Here's the quick test case I wrote (3 files):
c:\temp\dosprompt.rb
c:\temp\RunHide.vbs
c:\temp\test.bat

[dosprompt.rb code]
puts "Hi"
IO.popen("cmd /c c:/temp/test.bat")
sleep 2
puts "bye"
[/dosprompt.rb code]

[test.bat code]
echo hello > c:\asdfxxx.txt
[/test.bat code]

[RunHide.vbs code was previously mentioned above/]

To validate that it did in fact run, just run this from a command
prompt or from start > run
"WScript c:\temp\RunHide.vbs Cmd /C "C:\temp\dosprompt.rb""

To validate it ran ok...
c:\asdfxxx.txt should exist and contain "hello" :)

This seems to easy so.. perhaps I missed something.. if not, I'll be
happy about this tomorrow morning. Let me know if this works out for
you guys..

Credit to RunHide goes to this guy:
http://www.fpschultze.de/w5.htm

thx

[Simon Kröger]

stevetuckner wrote:
> [...]

> Below was my attempt to say, here is how you should fix this issue. And
> then it didn't work. If I run it with ruby it works fine. When I run it
> with rubyw, it just goes away when I click run. Any ideas anyone? The
> system code and `` code is taken from the list a long time ago.
>
> Steve Tuckner

> [...cool code...]

From the msdn library:

Note The _popen function returns an invalid file handle, if used in a
Windows program, that will cause the program to hang indefinitely.
_popen works properly in a Console application. To create a Windows
application that redirects input and output, read the section "Creating
a Child Process with Redirected Input and Output" in the Win32 SDK.

hmmm ... i will read through the mentioned section ....
(it's not 100% clear to me if this means "don't start a windows program
with popen" or "don't start any program from a windows program" - it
would make kind of sense if redirecting stdin from a program without
stdin would fail)

btw: on my computers (XP-Home) nothing 'goes away' but the cmd.exe
window flickers as if ruby's popen would be used - strange.

From the msdn again:

.and asynchronously executes a spawned copy of the command processor..

this is the main problem i guess.

cheers

Simon

[Simon Kröger]

Simon Kröger wrote:
> [...]

> hmmm ... i will read through the mentioned section ....

I did.

Ok, here is the 'solution': one has to use CreateProcess to spawn the
child. Unfortunately this is a bit more complicated but thanks to
Win32API its all achievable in pure ruby.

The stuff in $0 == __FILE__ builds a simple fox gui and starts cmd.exe
if a button is pressed. It uses a pipe to write to the stdin of the
child ('dir\n') and reads back the childs stdout via another pipe and
dumps it to a text control. (The stderror is also mapped but not used)

It's still some work to make it look as easy as popen, but definitaly
possible.

Here is the code:
(some lines are copied from this list and the net)

--------------------------------------------------------------------
require 'Win32API'

NORMAL_PRIORITY_CLASS = 0x00000020
STARTUP_INFO_SIZE = 68
PROCESS_INFO_SIZE = 16
SECURITY_ATTRIBUTES_SIZE = 12

ERROR_SUCCESS = 0x00
FORMAT_MESSAGE_FROM_SYSTEM = 0x1000
FORMAT_MESSAGE_ARGUMENT_ARRAY = 0x2000

HANDLE_FLAG_INHERIT = 1
HANDLE_FLAG_PROTECT_FROM_CLOSE =2

STARTF_USESHOWWINDOW = 0x00000001
STARTF_USESTDHANDLES = 0x00000100

def raise_last_win_32_error
errorCode = Win32API.new("kernel32", "GetLastError", [], 'L').call
if errorCode != ERROR_SUCCESS
params = [
'L', # IN DWORD dwFlags,
'P', # IN LPCVOID lpSource,
'L', # IN DWORD dwMessageId,
'L', # IN DWORD dwLanguageId,
'P', # OUT LPSTR lpBuffer,
'L', # IN DWORD nSize,
'P', # IN va_list *Arguments
]

formatMessage = Win32API.new("kernel32", "FormatMessage", params, 'L')
msg = ' ' * 255
msgLength = formatMessage.call(FORMAT_MESSAGE_FROM_SYSTEM +
FORMAT_MESSAGE_ARGUMENT_ARRAY, '', errorCode, 0, msg, 255, '')

msg.gsub!(/\000/, '')
msg.strip!
raise msg
else
raise 'GetLastError returned ERROR_SUCCESS'
end
end

def create_pipe # returns read and write handle
params = [
'P', # pointer to read handle
'P', # pointer to write handle
'P', # pointer to security attributes
'L'] # pipe size

createPipe = Win32API.new("kernel32", "CreatePipe", params, 'I')

read_handle, write_handle = [0].pack('I'), [0].pack('I')
sec_attrs = [SECURITY_ATTRIBUTES_SIZE, 0, 1].pack('III')

raise_last_win_32_error if createPipe.Call(read_handle,
write_handle, sec_attrs, 0).zero?

[read_handle.unpack('I')[0], write_handle.unpack('I')[0]]
end

def set_handle_information(handle, flags, value)
params = [
'L', # handle to an object
'L', # specifies flags to change
'L'] # specifies new values for flags

setHandleInformation = Win32API.new("kernel32",
"SetHandleInformation", params, 'I')
raise_last_win_32_error if setHandleInformation.Call(handle,
flags, value).zero?
nil
end

def close_handle(handle)
closeHandle = Win32API.new("kernel32", "CloseHandle", ['L'], 'I')
raise_last_win_32_error if closeHandle.call(handle).zero?
end

def create_process(command, stdin, stdout, stderror)
params = [
'L', # IN LPCSTR lpApplicationName
'P', # IN LPSTR lpCommandLine
'L', # IN LPSECURITY_ATTRIBUTES lpProcessAttributes
'L', # IN LPSECURITY_ATTRIBUTES lpThreadAttributes
'L', # IN BOOL bInheritHandles
'L', # IN DWORD dwCreationFlags
'L', # IN LPVOID lpEnvironment
'L', # IN LPCSTR lpCurrentDirectory
'P', # IN LPSTARTUPINFOA lpStartupInfo
'P'] # OUT LPPROCESS_INFORMATION lpProcessInformation

startupInfo = [STARTUP_INFO_SIZE, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
STARTF_USESTDHANDLES | STARTF_USESHOWWINDOW, 0,
0, 0, stdin, stdout, stderror].pack('IIIIIIIIIIIISSIIII')

processInfo = [0, 0, 0, 0].pack('IIII')
command << 0

createProcess = Win32API.new("kernel32", "CreateProcess", params, 'I')
raise_last_win_32_error if createProcess.call(0,
command, 0, 0, 1, 0, 0, 0, startupInfo, processInfo).zero?

hProcess, hThread, dwProcessId, dwThreadId = processInfo.unpack('LLLL')

close_handle(hProcess)
close_handle(hThread)

[dwProcessId, dwThreadId]
end

def write_file(hFile, buffer)
params = [
'L', # handle to file to write to
'P', # pointer to data to write to file
'L', # number of bytes to write
'P', # pointer to number of bytes written
'L'] # pointer to structure for overlapped I/O

written = [0].pack('I')
writeFile = Win32API.new("kernel32", "WriteFile", params, 'I')

raise_last_win_32_error if writeFile.call(hFile, buffer, buffer.size,
written, 0).zero?

written.unpack('I')
end

def read_file(hFile)
params = [
'L', # handle of file to read
'P', # pointer to buffer that receives data
'L', # number of bytes to read
'P', # pointer to number of bytes read
'L'] #pointer to structure for data

number = [0].pack('I')
buffer = ' ' * 255

readFile = Win32API.new("kernel32", "ReadFile", params, 'I')

return '' if readFile.call(hFile, buffer, 255, number, 0).zero?

buffer[0...number.unpack('I')[0]]
end

if $0 == __FILE__
require 'fox12'

include Fox

application = FXApp.new("popen", "popen")

main = FXMainWindow.new(application, "popen", nil, nil, DECOR_ALL,
0, 0, 500, 500, 10, 10, 10, 10, 10, 10)

button = FXButton.new(main, "&Do it!", nil, nil, 0,
FRAME_THICK|FRAME_RAISED|LAYOUT_FILL_X|LAYOUT_TOP|LAYOUT_LEFT,
0, 0, 0, 0, 10, 10, 5, 5)

frame = FXHorizontalFrame.new(main,
FRAME_THICK|FRAME_SUNKEN|LAYOUT_FILL_X|LAYOUT_FILL_Y,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0)

edit = FXText.new(frame, nil, 0,
LAYOUT_FILL_X|LAYOUT_FILL_Y|TEXT_READONLY, 0, 0, 0, 0)

edit.textColor, edit.backColor = 0xFFFFFF, 0

button.connect(SEL_COMMAND) do
cmd = 'cmd.exe'
input = "dir\nexit\n"

# create 3 pipes
child_in_r, child_in_w = create_pipe
child_out_r, child_out_w = create_pipe
child_error_r, child_error_w = create_pipe

# Ensure the write handle to the pipe for STDIN is not inherited.
set_handle_information(child_in_w, HANDLE_FLAG_INHERIT, 0)
set_handle_information(child_out_r, HANDLE_FLAG_INHERIT, 0)
set_handle_information(child_error_r, HANDLE_FLAG_INHERIT, 0)

processId, threadId = create_process(cmd, child_in_r,
child_out_w, child_error_w)

# we have to close the handles, so the pipes terminate with the process
close_handle(child_out_w)
close_handle(child_error_w)

write_file(child_in_w, input)

while !(buffer = read_file(child_out_r)).empty?
edit.appendText(buffer.gsub("\r", ''))
end
end

application.create()
main.show(PLACEMENT_SCREEN)
application.run()
end
----------------------------------------------------------------------

i would love to see this taken further, eventually in the popen
implementation?

cheers

Simon

[Simon Kröger]

x1 wrote:

> I just found a fix that works for me.. I hope this helps someone else out...

>
> Nonetheless, it does require a small vb script, which acts as a
> parent for the child script.. to execute, the command is as follows:
>
> [code]
> WScript RunHide.vbs Cmd /C MyBatch.cmd
> [/code]

Hmm, this might be a workaround but not realy a fix. At least not for my
problem. I don't want to distribute .vbs and .bat files.

Perhaps my other post offers a solution for you, too?

It's not very polished yet, but it does work (at least on my machine :))

cheers

Simon

[x1]

Simon, this worked on my machine also --excellent!

---Question / Request:
I tried stripping out the GUI stuff so that I could implement this
'solution' into my current needs (to no avail)

I would like to use this as a replacement to popen(), could you help
me strip this down into popen() form?

After the read_file method, I tried keeping what appeared to do the
actual work but.. no luck.. any suggestions?
#------------------------------------------------------------------------------------------------->
cmd = 'cmd.exe'
input = "c:\\temp\\test.bat\nexit\n"

# create 3 pipes
child_in_r, child_in_w = create_pipe
child_out_r, child_out_w = create_pipe
child_error_r, child_error_w = create_pipe
# Ensure the write handle to the pipe for STDIN is not inherited.
set_handle_information(child_in_w, HANDLE_FLAG_INHERIT, 0)
set_handle_information(child_out_r, HANDLE_FLAG_INHERIT, 0)
set_handle_information(child_error_r, HANDLE_FLAG_INHERIT, 0)

processId, threadId = create_process(cmd, child_in_r,
child_out_w, child_error_w)

# we have to close the handles, so the pipes terminate with the process
close_handle(child_out_w)
close_handle(child_error_w)

write_file(child_in_w, input)
#------------------------------------------------------------------------------------------------->
# while !(buffer = read_file(child_out_r)).empty?
# edit.appendText(buffer.gsub("\r", ''))
# end

thank you very much!

[Simon Kröger]

x1 wrote:

> Simon, this worked on my machine also --excellent!
>
> ---Question / Request:
> I tried stripping out the GUI stuff so that I could implement this
> 'solution' into my current needs (to no avail)
>
> I would like to use this as a replacement to popen(), could you help
> me strip this down into popen() form?
>
> After the read_file method, I tried keeping what appeared to do the
> actual work but.. no luck.. any suggestions?

ok, here we go:
---------------------------------------
io = popen('ver')
puts io.read_all
---------------------------------------
better?

or:
---------------------------------------
io = popen('cmd')
io.write("ping localhost\nexit\n")

while !(buffer = io.read).empty?
print buffer
end
---------------------------------------
?

note that read blocks if there is no data to read, but this is done in a
ruby-thread friendly way. So you may put that in a new Thread and your
gui is still responsive (i tried that with the fox gui).
(if you let it block on read_file the whole process blocks)

The same is true for read_all and you can have multiple childs at once.

here is the complete code:

-------------------------------------------------------------------------
require 'Win32API'

$stdout.sync = true

close_handle(hProcess)
close_handle(hThread)

[dwProcessId, dwThreadId]
end

written.unpack('I')[0]
end

def read_file(hFile)
params = [
'L', # handle of file to read
'P', # pointer to buffer that receives data
'L', # number of bytes to read
'P', # pointer to number of bytes read
'L'] #pointer to structure for data

number = [0].pack('I')
buffer = ' ' * 255

readFile = Win32API.new("kernel32", "ReadFile", params, 'I')
return '' if readFile.call(hFile, buffer, 255, number, 0).zero?

buffer[0...number.unpack('I')[0]]
end

def peek_named_pipe(hFile)
params = [
'L', # handle to pipe to copy from
'L', # pointer to data buffer
'L', # size, in bytes, of data buffer
'L', # pointer to number of bytes read
'P', # pointer to total number of bytes available
'L'] # pointer to unread bytes in this message

available = [0].pack('I')
peekNamedPipe = Win32API.new("kernel32", "PeekNamedPipe", params, 'I')

return -1 if peekNamedPipe.Call(hFile, 0, 0, 0, available, 0).zero?

available.unpack('I')[0]
end

class Win32popenIO
def initialize (hRead, hWrite)
@hRead = hRead
@hWrite = hWrite
end

def write data
write_file(@hWrite, data.to_s)
end

def read
sleep(0.01) while peek_named_pipe(@hRead).zero?
read_file(@hRead)
end

def read_all
all = ''
while !(buffer = read).empty?
all << buffer
end
all
end
end

def popen(command)

# create 3 pipes
child_in_r, child_in_w = create_pipe
child_out_r, child_out_w = create_pipe
child_error_r, child_error_w = create_pipe

# Ensure the write handle to the pipe for STDIN is not inherited.
set_handle_information(child_in_w, HANDLE_FLAG_INHERIT, 0)
set_handle_information(child_out_r, HANDLE_FLAG_INHERIT, 0)
set_handle_information(child_error_r, HANDLE_FLAG_INHERIT, 0)

processId, threadId = create_process(ENV['ComSpec'] + ' /C ' +
command, child_in_r, child_out_w, child_error_w)

# we have to close the handles, so the pipes terminate with the process

close_handle(child_in_r)
close_handle(child_out_w)
close_handle(child_error_w)

Win32popenIO.new(child_out_r, child_in_w)
end

if $0 == __FILE__
io = popen('ver')
puts io.read_all

io = popen('cmd')
io.write("ping localhost\nexit\n")

while !(buffer = io.read).empty?
print buffer.gsub("\r", '')
end
end
-------------------------------------------------------------------------

btw: i would love to have all the methods from IO but i realy don't like
to implement them all - there must be another way.

cheers

Simon

[x1]

This works like a champ. Thank you again --Fantastic!

Robert/Alex/Steve, does this do the trick for you?

Kind regards,
Chris