Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Problem with redirection of command output in ooREXX V400 on Win XP Pro.

62 views
Skip to first unread message

Jeremy Nicoll - news posts

unread,
May 20, 2012, 2:59:51 PM5/20/12
to
Using ooREXX V4.0.0 on Win XP Pro

I have an existing home-grown function I use for wrapping redirection of
stdout (and sometimes stderr) when I call a command-line program under
Windows XP Pro. Usually the programs I call are either part of Windows or
utilities I've placed in a directory which is on PATH.

In execs where I plan to use such things I normally start by issuing a test
form of the command to check I get an expected simple response.

So for example with a utility called "id3.exe" I'll predefine the name of a
temporary file then do something like:

cmd = "id3 -V >" tempfile /* get version-no of id3.exe */
cmd ; cmdrc = rc

and go on to process the contents of the temporary file. The temporary
files I use often have spaces in their names so actually I create cmd as:

cmd = "id3 -V >" '"'tempfile'"'


Yesterday I tried to call 'curl' this way. However my copy of curl.exe is
not in a directory that's on PATH, because it and several other programmes
I'd like to have on PATH use different versions of some standard DLLs. If I
put any one set of utility programs and its DLLs into a directory on PATH
then I can't put other utility programs that'd use the same-named DLLs into
the same directories... so I've chosen to keep all of them elsewhere.
Anyway I ended up with a command string like:

cmd = '"C:\full\path\to\curl.exe" -V > "C:\full\pathto\tempfile.txt"'

and... it didn't work. No output was written to the tempfile. But if I c&p
the command that was in cmd into a (cmd.exe) command window, it does work.

For ages I thought this was a curl problem, then (when I tried other
utilities' names instead of curl.exe) I thought it was a Windows problem
(but the failing commands always work in command windows), so now I think
it's an ooREXX problem.


I wrote a test program to issue commands in various slightly different ways;
it logs the exact commands it issues and the sizes of the temporary file
used for each attempt (and the size of a file that's written to if a test
file name isn't enclosed in quotes), and also the contents of both those
files after issuing each command. It summarises command, rc, file
existence/sizes for each attempt. So issuing

edlin /?

in various ways and redirecting output I get:


#ec=01 TM=0005, C-VF=NO for: edlin /? > C:\vv1-cmd1a.txt

#ec=02 TM=0005, C-VF=NO for: edlin /? > "C:\vv1-cmd1b.txt"

#ec=03 TMPF=NO, CV=0005 for: edlin /? > C:\v v\1-cmd1c.txt

#ec=04 TM=0005, C-VF=NO for: edlin /? > "C:\v v\1-cmd1d.txt"


#ec=05 TM=0005, C-VF=NO for: edlin.exe /? > C:\vv1-cmd2a.txt

#ec=06 TM=0005, C-VF=NO for: edlin.exe /? > "C:\vv1-cmd2b.txt"

#ec=07 TMPF=NO, CV=0005 for: edlin.exe /? > C:\v v\1-cmd2c.txt

#ec=08 TM=0005, C-VF=NO for: edlin.exe /? > "C:\v v\1-cmd2d.txt"

#ec=09 TM=0005, C-VF=NO for: "C:\WINDOWS\system32\edlin.exe" /? >
C:\vv1-cmd3a.txt

#ec=10 TMPF=NO, C-VF=NO for: "C:\WINDOWS\system32\edlin.exe" /? >
"C:\vv1-cmd3b.txt"

#ec=11 TMPF=NO, CV=0005 for: "C:\WINDOWS\system32\edlin.exe" /? > C:\v
v\1-cmd3c.txt

#ec=12 TMPF=NO, C-VF=NO for: "C:\WINDOWS\system32\edlin.exe" /? > "C:\v
v\1-cmd3d.txt"


In each of these log summary lines the TMPF=NO or TM=nnnn value says whether
the expected tempfile was not created, or had nnnn (0 or more) lines. The
similar C-VF=NO or CV=nnnn value refers to a file called "C:\v" which is
accidentally created and written to when a tempfile with a name like



C:\v v\1-cmd2c.txt

is not wrapped up in quotes properly. The 12 tests above show that edlin /?
creates 5 lines of output in the expected temporary file or the accidentally
specified one for 10 of the 12 tests. The remaining two tests generated no
output at all, that is, for command strings containing:

"C:\WINDOWS\system32\edlin.exe" /? > "C:\vv1-cmd3b.txt"

"C:\WINDOWS\system32\edlin.exe" /? > "C:\v v\1-cmd3d.txt"


When either of these commands is c&p into a command window, it works.
Note that the double-quotes shown are all part of the commands.


It seems to me that the combination of enclosing the .exe's full path in
quotes AND enclosing the temporary file's full path in quotes, causes a
problem when the command is issued from ooREXX.

Does anyone have any idea what's going on, before I raise a bug report?


I've done the same thing with the same behaviour for 4 separate utility
programs (mainly ones that normally process MP3 files), and curl, as well as
for "edlin" and "ipconfig" which are part of windows, and it seems to me to
be the same behaviour in each case.


Here's the same thing for: ipconfig /?


#ec=01 TM=0039, C-VF=NO for: ipconfig /? > C:\vv2-cmd1a.txt

#ec=02 TM=0039, C-VF=NO for: ipconfig /? > "C:\vv2-cmd1b.txt"

#ec=03 TMPF=NO, CV=0039 for: ipconfig /? > C:\v v\2-cmd1c.txt

#ec=04 TM=0039, C-VF=NO for: ipconfig /? > "C:\v v\2-cmd1d.txt"

#ec=05 TM=0039, C-VF=NO for: ipconfig.exe /? > C:\vv2-cmd2a.txt

#ec=06 TM=0039, C-VF=NO for: ipconfig.exe /? > "C:\vv2-cmd2b.txt"

#ec=07 TMPF=NO, CV=0039 for: ipconfig.exe /? > C:\v v\2-cmd2c.txt

#ec=08 TM=0039, C-VF=NO for: ipconfig.exe /? > "C:\v v\2-cmd2d.txt"

#ec=09 TM=0039, C-VF=NO for: "C:\WINDOWS\system32\ipconfig.exe" /? >
C:\vv2-cmd3a.txt
#ec=10 TMPF=NO, C-VF=NO for: "C:\WINDOWS\system32\ipconfig.exe" /? >
"C:\vv2-cmd3b.txt"
#ec=11 TMPF=NO, CV=0039 for: "C:\WINDOWS\system32\ipconfig.exe" /? > C:\v
v\2-cmd3c.txt
#ec=12 TMPF=NO, C-VF=NO for: "C:\WINDOWS\system32\ipconfig.exe" /? > "C:\v
v\2-cmd3d.txt"


--
Jeremy C B Nicoll - my opinions are my own.

Email sent to my from-address will be deleted. Instead, please reply
to newsre...@wingsandbeaks.org.uk replacing "aaa" by "284".

Jeremy Nicoll - news posts

unread,
May 20, 2012, 3:13:17 PM5/20/12
to
Jeremy Nicoll - news posts <jn.nntp....@wingsandbeaks.org.uk> wrote:

>I have an existing ...

I forgot to add: the rc from the executed command varies, depending as far
as I can see on the utility itself. In the "edlin /?" example, I get rc=0
for the 10 commands that worked and rc=1 for the two that didn't.

But for "ipconfig /?" I get rc=1 for all 12, even though ten of those worked
and two didn't.

Swifty

unread,
May 22, 2012, 2:31:11 AM5/22/12
to
On Sun, 20 May 2012 19:59:51 +0100, Jeremy Nicoll - news posts
<jn.nntp....@wingsandbeaks.org.uk> wrote:

>It seems to me that the combination of enclosing the .exe's full path in
>quotes AND enclosing the temporary file's full path in quotes, causes a
>problem when the command is issued from ooREXX.

For what it's worth, I cannot recreate this with 4.1.1

However, it's an ill wind that blows no good. Your problem highlighted
the fact that the Command() function in my common subroutine library
would fail if there were blanks in either the command or the temporary
file name, so I'll correct it always to enclose both in double quotes.

--
Steve Swift
http://www.swiftys.org.uk/swifty.html
http://www.ringers.org.uk

Jeremy Nicoll - news posts

unread,
May 22, 2012, 4:26:49 AM5/22/12
to
Swifty <steve....@gmail.com> wrote:

>For what it's worth, I cannot recreate this with 4.1.1

It's worth a lot; it might be the kick in the pants I need to upgrade. But
I hate doing things like that without having a few days (because I can only
do a wee bit each day) to sort out problems with an upgrade, and just at the
moment things are piling up.

> However, it's an ill wind that blows no good. Your problem highlighted the
> fact that the Command() function in my common subroutine library would
> fail if there were blanks in either the command or the temporary file
> name, so I'll correct it always to enclose both in double quotes.

And every/any other file handling process, I hope.

Swifty

unread,
May 22, 2012, 1:08:33 PM5/22/12
to
On Tue, 22 May 2012 09:26:49 +0100, Jeremy Nicoll - news posts
<jn.nntp....@wingsandbeaks.org.uk> wrote:

>And every/any other file handling process, I hope.

Yes, and so do I.

My Command() function wasn't as daft as I'd feared. If I pass a
command to it where the executable contains a space, it's the caller's
responsibility to quote the executable, as only they know which spaces
are part of the filename, and which space is the delimiter between the
executable and its parameters.

Also, this subroutine is on my PC only, and the temporary file will
never have a space in it as long as my TEMP environment variable
contains no spaces.

So the exposure was very small.

Jeremy Nicoll - news posts

unread,
May 22, 2012, 1:41:42 PM5/22/12
to
Swifty <steve....@gmail.com> wrote:

>On Tue, 22 May 2012 09:26:49 +0100, Jeremy Nicoll - news posts
><jn.nntp....@wingsandbeaks.org.uk> wrote:
>
>>And every/any other file handling process, I hope.
>
>Yes, and so do I.
>
> My Command() function wasn't as daft as I'd feared. If I pass a command to
> it where the executable contains a space, it's the caller's responsibility
> to quote the executable, as only they know which spaces are part of the
> filename, and which space is the delimiter between the executable and its
> parameters.

My tendency is to put quotes around anything that might need it, so all
filenames everywhere (in commands anyway). In calling 'curl.exe' there's
lots of parameters that might need quoted - filenames, URLs (eg if they
contain an ampersand), inline urlencoded form data (percent signs, I think
are the problem) and so on.

I finally got my test code to work by using a copy of curl.exe (and its
DLLs) in a directory on PATH so my commands now all start with "curl "
rather than its quoted path, then every parameter along the command line is
quoted if it may need it, and lastly the redirection file is not enclosed in
quotes - which is fine if it's in TEMP but makes it impossible to put such a
file anywhere much else. This will do as a workaround, and stopped me going
mad thinking curl no longer worked, but it makes it impossible to do the
same with other .exe's using the same-named (but different content) DLLs.

Rugxulo

unread,
May 22, 2012, 2:11:46 PM5/22/12
to
Hi,

On May 22, 12:41 pm, Jeremy Nicoll - news posts
<jn.nntp.scrap...@wingsandbeaks.org.uk> wrote:
> Swifty <steve.j.sw...@gmail.com> wrote:
> >On Tue, 22 May 2012 09:26:49 +0100, Jeremy Nicoll - news posts
> ><jn.nntp.scrap...@wingsandbeaks.org.uk> wrote:
>
> >>And every/any other file handling process, I hope.
>
> >Yes, and so do I.
>
> > My Command() function wasn't as daft as I'd feared. If I pass a command to
> > it where the executable contains a space, it's the caller's responsibility
> > to quote the executable, as only they know which spaces are part of the
> > filename, and which space is the delimiter between the executable and its
> > parameters.
>
> My tendency is to put quotes around anything that might need it, so all
> filenames everywhere (in commands anyway).  In calling 'curl.exe' there's
> lots of parameters that might need quoted - filenames, URLs (eg if they
> contain an ampersand), inline urlencoded form data (percent signs, I think
> are the problem) and so on.
>
> I finally got my test code to work by using a copy of curl.exe (and its
> DLLs) in a directory on PATH so my commands now all start with "curl "
> rather than its quoted path, then every parameter along the command line is
> quoted if it may need it, and lastly the redirection file is not enclosed in
> quotes - which is fine if it's in TEMP but makes it impossible to put such a
> file anywhere much else.  This will do as a workaround, and stopped me going
> mad thinking curl no longer worked, but it makes it impossible to do the
> same with other .exe's using the same-named (but different content) DLLs.

A few minor comments, though it probably won't help.

1). XP is a bit different than later Windows, so you should be aware
of that. In particular, I think later versions aren't Admin by
default, so they won't let you write to C:\ without permission. (Also,
edlin is a DOS app and thus not supported in 64-bit versions.
Similarly for debug. So don't rely on those too heavily.)

2). I don't know why you insist upon spaces in filenames. For system
directories and installed apps, I know it's (mostly) out of your
hands, but on your end that kind of thing should be avoided if
possible. I know some people may dislike hearing that, but the truth
is that some tools just don't handle spaces very well. A dopey
workaround, assuming you haven't done "disable8dot3", would be to use
"for %a ... %~%sa" to use the SFN of the file / dir / etc. But for
temporary files, there must be a better way, even if using the CMD
built-in pseudo-var %RANDOM%.

3). CMD will let you quote certain chars (ampersand '&') with the
caret '^' if you need it. So if clash (due to special chars used by
the shell) is your only problem, you can forget quotes and just use
that.

4). If curl's .DLLs get in the way, just get (or make) a static
compile. Dunno about Cygwin, maybe not there, maybe MinGW (assuming it
is supported) or OpenWatcom would work. Heck, you could probably use
SwsVpkt (emulated packet driver) and a DJGPP version, if needed, at
least on XP.

Jeremy Nicoll - news posts

unread,
May 23, 2012, 7:42:24 AM5/23/12
to
Rugxulo <rug...@gmail.com> wrote:

> 1). XP is a bit different than later Windows, so you should be aware of
> that. In particular, I think later versions aren't Admin by default, so
> they won't let you write to C:\ without permission. (Also, edlin is a DOS
> app and thus not supported in 64-bit versions. Similarly for debug. So
> don't rely on those too heavily.)

I only chose C-root files as destinations so I'd have simple and short
filenames.

Edlin & ipconfig I used in examples because I'd expect every Windows version
to have them. The actual commands I've been having problems with are other
utilities that anyone wanting to duplicate my problem with would not want to
have to download/install. Showing the problem happens with 'proper' parts
of Windows rather than other people's weird utilities seemed like a good
idea.


> 2). I don't know why you insist upon spaces in filenames. For system
> directories and installed apps, I know it's (mostly) out of your hands,

Most aof my files are in C:\My Dropbox\... so there's no choice.

> but on your end that kind of thing should be avoided if possible.

Why's that?

> I know some people may dislike hearing that, but the truth is that some
> tools just don't handle spaces very well. A dopey workaround, assuming you
> haven't done "disable8dot3", would be to use "for %a ... %~%sa"

Pardon? That's not rexx.

> to use the SFN of the file / dir / etc. But for temporary files, there
> must be a better way, even if using the CMD built-in pseudo-var %RANDOM%.

I see no point in using a random number when a timestamp in a filename makes
it easy to sort by time of use.


> 3). CMD will let you quote certain chars (ampersand '&') with the caret
> '^' if you need it. So if clash (due to special chars used by the shell)
> is your only problem, you can forget quotes and just use that.

Ooh, I didn't know about "^"; very useful. Thanks!


> 4). If curl's .DLLs get in the way, just get (or make) a static compile.
> Dunno about Cygwin, maybe not there, maybe MinGW (assuming it is
> supported) or OpenWatcom would work. Heck, you could probably use SwsVpkt
> (emulated packet driver) and a DJGPP version, if needed, at least on XP.

I've never compiled anything under Windows (done lots under MVS though),
wouldn't know where to start. I just downloaded the Win32 version of curl
from the project website. As for Cygwin, it's 30+ years since I last used a
unix system and I don't want to make Windows more complicated than it needs
to be.

Rugxulo

unread,
May 23, 2012, 10:58:57 AM5/23/12
to
Hi,

On May 23, 6:42 am, Jeremy Nicoll - news posts
<jn.nntp.scrap...@wingsandbeaks.org.uk> wrote:
> Rugxulo <rugx...@gmail.com> wrote:
> > 1). XP is a bit different than later Windows, so you should be aware of
> > that. In particular, I think later versions aren't Admin by default, so
> > they won't let you write to C:\ without permission. (Also, edlin is a DOS
> > app and thus not supported in 64-bit versions. Similarly for debug. So
> > don't rely on those too heavily.)
>
> I only chose C-root files as destinations so I'd have simple and short
> filenames.

I understand, but writing to the root dir is generally frowned upon,
even if MSVCRT.DLL's tmpfile() also (incorrectly) attempts that, too.
In fact, several tools assume you can write there, which is annoying.
And on my laptop, some (OEM?) util put a random .DLL there, but I'm
not dangerous enough to randomly remove it, heh.

> Edlin & ipconfig I used in examples because I'd expect every Windows version
> to have them. The actual commands I've been having problems with are other
> utilities that anyone wanting to duplicate my problem with would not want to
> have to download/install.  Showing the problem happens with 'proper' parts
> of Windows rather than other people's weird utilities seemed like a good
> idea.

Yes, that makes sense, just wanted to mention that Win64 lacked it.
Though if really desperate for it, you could compile FreeDOS' edlin,
it's portable C (ironically enough).

> > 2). I don't know why you insist upon spaces in filenames. For system
> > directories and installed apps, I know it's (mostly) out of your hands,
>
> Most aof my files are in C:\My Dropbox\...  so there's no choice.

I figured, just always hear various warnings from various tools:
"don't do it!"

> > but on your end that kind of thing should be avoided if possible.
>
> Why's that?

Because some tools choke on spaces, whether passed as parameter or
even when themselves residing in such a dir. It's probably their bug,
but when enough projects suffer from it, it's not worth the risk, IMO.

> > I know some people may dislike hearing that, but the truth is that some
> > tools just don't handle spaces very well. A dopey workaround, assuming you
> > haven't done "disable8dot3", would be to use "for %a ... %~%sa"
>
> Pardon?  That's not rexx.

I know, but it's a valid workaround (maybe) if you're willing to use
CMD (which is default shell anyways on XP).

> > to use the SFN of the file / dir / etc. But for temporary files, there
> > must be a better way, even if using the CMD built-in pseudo-var %RANDOM%.
>
> I see no point in using a random number when a timestamp in a filename makes
> it easy to sort by time of use.

I just wanted to mention that even simple .BAT scripts can use random
numbers, if necessary.

> > 3). CMD will let you quote certain chars (ampersand '&') with the caret
> > '^' if you need it. So if clash (due to special chars used by the shell)
> > is your only problem, you can forget quotes and just use that.
>
> Ooh, I didn't know about "^"; very useful.  Thanks!

Yes, and while it's a nice feature, it sometimes rarely bites older
DOS .BATs that didn't have such a feature.

> > 4). If curl's .DLLs get in the way, just get (or make) a static compile.
> > Dunno about Cygwin, maybe not there, maybe MinGW (assuming it is
> > supported) or OpenWatcom would work. Heck, you could probably use SwsVpkt
> > (emulated packet driver) and a DJGPP version, if needed, at least on XP.
>
> I've never compiled anything under Windows (done lots under MVS though),
> wouldn't know where to start.

MinGW has an installer now too. So it should be fairly easy to grab
and use, but I'll admit to not having played with that. OpenWatcom is
easier (for me), but I've never compiled curl with it.

> I just downloaded the Win32 version of curl
> from the project website.

I know, and you mentioned .DLL problems. I wasn't sure if that was
complicating or breaking things for you. The answer is obvious (static
linking), but I don't know if that's easy or even possible in this
particular case. Sometimes what is possible or "best" isn't fully
supported.

> As for Cygwin, it's 30+ years since I last used a
> unix system and I don't want to make Windows more complicated than it needs
> to be.

I actually like compiling stuff, but yeah, *nix builds are often
annoying. And I'm no heavy Cygwin user. Like I said, curl already
exists for DJGPP (slightly older version, I think), and I know people
claim SwsVpkt works on XP, so that would be a workaround. (If I still
had XP, I'd do that, but I have lots of DJGPP stuff, so for me that
would be beneficial. For just a one-off, it may seem weird.)
0 new messages