ooRexx 4.0.0 Trapping command output
Jeremy Nicoll - news posts
My impression is that to trap command output there's two approaches:
a) command | temporaryfile (or command > temporaryfile)
read whole file into a stem (or array), in one operation
b) command | rxqueue
loop through queue a line at a time
I have a feeling that (a) is simpler especially if (eg if the command fails)
you want to display the whole command & its output to the user (as you can
directly open the temporary file in an editor for the user to do with
whatever they want). On the other hand, without ram disk, it could be a bit
slow as real disk I/O has to occur to put data into the file and read it out
again. Having the data end up in a stem is useful because you don't have to
read through that sequentially if you know its layout...
I like the elegance of (b) right up to the point where I need to put the
looping code into my own rexx exec. I have to read the whole contents of
the queue to get access to all its content. The greater the number of lines
of output the slower that's going to be...
I wish there was a filter program like rxqueue that put whatever it was
given into a named stem:
command | rxstem cmdout.
would be very useful.
Apart from command output, is there a way to get the return-code from the
command? I did wonder if one wrote a rexx exec 'docommand' that contains
parse arg cmdtoissue
cmdtoissue
cmdrc = rc
queue "cmd rc was" cmdrc
and then called it elsewhere by eg
rexx docommand "my command to be issued" | rxqueue
whether you'd then have access to the command rc as well as its results.
(One might need to use two specific named queues - one for the rc and one
for the results, for ease of use.)
What do other people do?
--
Jeremy C B Nicoll - my opinions are my own.
Email sent to my from-address will be deleted. Instead, please reply
to newsre...@wingsandbeaks.org.uk replacing "nnn" by "284".
Bob Martin
>ooRexx 4.0.0, Windows XP Pro
>
>My impression is that to trap command output there's two approaches:
>
>a) command | temporaryfile (or command > temporaryfile)
>read whole file into a stem (or array), in one operation
How do you read a file into a stem? I'd like to do that but haven't found out how!
Jeremy Nicoll - news posts
Ah, maybe you can't. Being a classic-rexx user of old I've tended to think
of "stem" and "stem array" and "array" as interchangeable terms, but I have
a feeling that nowadays "array" might (or maybe not?) mean something else.
Nevertheless if you look for "makearray", "arrayIn", "arrayOut" in the
reference manual you'll see that there's some sort of single-operation
available, but into or out of "arrays".
There's also SysFileSearch() which is documented as loading all the lines of
a file which contain a specified search string into a /stem/. I'm kind of
hoping that
SysFileSearch("",filename,stemname)
might load every line, on the assumption that "" will occur on every line.
Swifty
> I have a feeling that (a) is simpler especially if (eg if the command fails)
> directly open the temporary file in an editor for the user to do with
> whatever they want). On the other hand, without ram disk, it could be a bit
> slow as real disk I/O has to occur to put data into the file and read it out
> again.
You must benchmark your assumption that file I/O will be slower than
RXQUEUE � in the environment where your code will run.
I have done this for every environment that I encounter, and so far the
file I/O is winning, even when there is no ram disk. Given that RXQUEUE
works entirely in storage, this is a very unexpected result!
You also have to be very careful with RXQUEUE if your REXX is ever
likely to get multi-threaded (as in CGI scripts all running under
apache) � the threads can corrupt each others queues. I may have
confused threads with processes here, but you get the idea.
I put all such routines into my subroutine library. I can send it to you
if you like.
Command() � returns a stem of the output from the command.
stem.0rc is the return code
This uses:
Records() � reads a file into a stem. Stem.0rc is the return code
and stem.0msg is the error message
This uses:
Fileread() � reads a file in one read; returns RC and DATA
--
Steve Swift
http://www.swiftys.org.uk/swifty.html
http://www.ringers.org.uk
Jeremy Nicoll - news posts
> You must benchmark your assumption that file I/O will be slower than
> RXQUEUE � in the environment where your code will run.
>
> I have done this for every environment that I encounter, and so far the
> file I/O is winning, even when there is no ram disk. Given that RXQUEUE
> works entirely in storage, this is a very unexpected result!
Certainly is! Is that true for an exec's session queue, or only a problem
when named global queues are used (says he, assuming that the separate rxapi
task might be involved only for the global ones?)?
> You also have to be very careful with RXQUEUE if your REXX is ever
> likely to get multi-threaded (as in CGI scripts all running under
> apache) � the threads can corrupt each others queues. I may have
> confused threads with processes here, but you get the idea.
I do indeed. I was thinking the 'external data queue' doc was a bit sparse
on detail about precisely how this was prevented; it seemed to me that with
an option that a particular exec can use to specify the queue it's about to
use for some processing, there might be a problem if some other exec changed
that setting while the first was processing.
I suppose you could wrap queue access up in mutex semaphores, but that only
works if every exec that does so uses the same logic, which I could ensure
for my code, but not so readily for any code that came from anywhere else.
> I put all such routines into my subroutine library. I can send it to you
> if you like.
>
> Command() � returns a stem of the output from the command.
> stem.0rc is the return code
> This uses:
>
> Records() � reads a file into a stem. Stem.0rc is the return code
> and stem.0msg is the error message
> This uses:
>
> Fileread() � reads a file in one read; returns RC and DATA
Yes please. Please note what my sig says!
bwcherry
<jn.nntp.scrap...@wingsandbeaks.org.uk> wrote:
> Swifty <steve.j.sw...@gmail.com> wrote:
> > You must benchmark your assumption that file I/O will be slower than
>
> > I have done this for every environment that I encounter, and so far the
> > file I/O is winning, even when there is no ram disk. Given that RXQUEUE
> > works entirely in storage, this is a very unexpected result!
>
> Certainly is! Is that true for an exec's session queue, or only a problem
> when named global queues are used (says he, assuming that the separate rxapi
> task might be involved only for the global ones?)?
>
> > You also have to be very careful with RXQUEUE if your REXX is ever
> > likely to get multi-threaded (as in CGI scripts all running under
> > confused threads with processes here, but you get the idea.
>
> I do indeed. I was thinking the 'external data queue' doc was a bit sparse
> on detail about precisely how this was prevented; it seemed to me that with
> an option that a particular exec can use to specify the queue it's about to
> use for some processing, there might be a problem if some other exec changed
> that setting while the first was processing.
>
> I suppose you could wrap queue access up in mutex semaphores, but that only
> works if every exec that does so uses the same logic, which I could ensure
> for my code, but not so readily for any code that came from anywhere else.
>
> > I put all such routines into my subroutine library. I can send it to you
> > if you like.
>
> > stem.0rc is the return code
> > This uses:
>
> > and stem.0msg is the error message
> > This uses:
>
>
> Yes please. Please note what my sig says!
>
> --
> Jeremy C B Nicoll - my opinions are my own.
>
> Email sent to my from-address will be deleted. Instead, please reply
You might want to check out "The RexxQueue Class" to get around the
multi-threading issues of "rxqueue('set', queuename)". (works for me)
If you are using ooRexx 4 you can do the following:
'set | rxqueue'
lines = .stdque~makearray()
do line over lines
say line
end
--
Brandon Cherry
Mark Miesfeld
<jn.nntp.scrap...@wingsandbeaks.org.uk> wrote:
I just read this whole thread and have a few comments. It's hard to
pick a good place to interject them, so I'm just arbitrarily picking
this one.
> Ah, maybe you can't. Being a classic-rexx user of old I've tended to think
> of "stem" and "stem array" and "array" as interchangeable terms, but I have
> a feeling that nowadays "array" might (or maybe not?) mean something else.
As an aside, in ooRexx, an array object and a stem object are
different. I'd use the arrrayIn() method of the Stream class if I
wanted to read the file into an object.
I mostly wanted to comment on rxqueue. This is a separate
executable. Because of that you will always lose the return code of
whatever the "command" is that you are issuing and want to capture the
output of. rxqueue runs last, so RC will always be the return code
rxqueue.
Because of that, I'd always redirect output into a file.
rxqueue reads its input one character at a time, that's probably why
it's slower in Steve Swift's benchmarks.
--
Mark Miesfeld
Arthur T.
Message-ID:<gemini.kuisw...@wingsandbeaks.org.uk.invalid>,
Jeremy Nicoll - news posts <jn.nntp....@wingsandbeaks.org.uk>
wrote:
>ooRexx 4.0.0, Windows XP Pro
>
>My impression is that to trap command output there's two approaches:
>
>a) command | temporaryfile (or command > temporaryfile)
> read whole file into a stem (or array), in one operation
>
>b) command | rxqueue
> loop through queue a line at a time
>
Some people have said that ooREXX is approximately a superset
of what's in Regina. Regina has the option to issue a command and
get the output into a stem. One of its examples is:
ADDRESS SYSTEM "sort" WITH INPUT STEM names. OUTPUT STEM names.
The Regina manual also states that this syntax matches ANSI
specs.
Does ooREXX have this capability? If so, that should certainly
satisfy the OP's request.
--
Arthur T. - ar23hur "at" intergate "dot" com
Looking for a z/OS (IBM mainframe) systems programmer position
Jeremy Nicoll - news posts
> Does ooREXX have this capability? If so, that should certainly
> satisfy the OP's request.
If it does, I've found nothing about it in the manuals.
All I want really is OUTTRAP().
Jeremy Nicoll - news posts
> You might want to check out "The RexxQueue Class" to get around the
> multi-threading issues of "rxqueue('set', queuename)". (works for me)
I've read as much as I could on this, but found it insufficiently detailed.
The reference manual section on "The RexxQueue Class" gives no info at all
apart from very basic syntax / actions of the calls.
I find it hard to believe that the class would not just use the same
underlying code as the rxqueue() function - why would the implementers write
two versions of the same thing? And worse, if they have done, why not
document the differences?
Section 14.1.3 on external queues describes
previousqname = rxqueue("set",newqueuename)
but does not say whether the name that becomes set is unique to the context
(if that's the word) of the exec that set it, or whether it's a globally set
value for "the current queue to use if no specific one is specified" by any
exec executing anywhere on the machine until further notice.
I also wondered if, if the exec that set a queuename crashes whether in
cleanup the previously in-use default queuename gets restored or what?
(Obviously in non-crash error handling one would probably reset whatever
queuename had been set before one's own exec got control.)
I had considered having any exec of mine that wanted to use an external
queue creating one with a unique name (ie something made up from its process
id & thread id & date & time of creation). That's probably good practice
anyway.
But it's moot if some other exec overrides the 'set' value.
I see that the RXQUEUE filter will use the queue named by the RXQUEUE
environment variable if an explicit name isn't provided to it. In
Windows/Linux, I assume there are usually different sets of environment vars
per process, but not for multiple threads within a process. Is that right?
If so that makes "cmd | RXQUEUE" useless, though "cmd | RXQUEUE qname"
would be useable provided while that specific filter is running it doesn't
change the name of the queue it's writing to even if the environment
variable's value changes in that time.
I also wondered about wrapping rxqueue 'set' commands in a mutex, which -
again - would be ok for all my own code, but useless if some other exec
(especially a compiled one where I couldn't examine its code) got run on my
system.
> If you are using ooRexx 4 you can do the following:
>
> 'set | rxqueue'
> lines = .stdque~makearray()
> do line over lines
> say line
> end
- but unfortunately that tramples all over (or at least appends stuff to)
the current contents of the SESSION queue. I can think of situations where
I'd not want that.
--
Jeremy C B Nicoll - my opinions are my own.
Email sent to my from-address will be deleted. Instead, please reply
to newsre...@wingsandbeaks.org.uk replacing "nnn" by "284".
bwcherry
<jn.nntp.scrap...@wingsandbeaks.org.uk> wrote:
WARNING! This info is based off of my usage of the interpreter and I
have not reviewed all of the code in question. I'm sure Rick McGuire
or Mark Miesfeld will jump in to correct me.
To demonstrate the limitation with setting the queue, try reading from
multiple commands at the same time with oorexx's built-in
multithreading capabilities (ex: "early reply" that's how I found this
info on pre 3.2.0). I was not able to find a way to read from more
than queue at the exact same time. The problem with the BIFs is they
are using (from what I can gather) a single instance of the rxqueue
class. Problems can be introduced when you set the queue, it changes
for all threads in the same instance of the interpreter. This does not
apply if you are running multiple instances of the interpreter.
Some of the other problems with queues are the names are system wide.
You will need to make sure your queuename is unique enough. For a
small script that might be not be a problem but not for something
running from say a server. The session queue is automatically
generated to be unique enough to the system for each interpreter
instance (environment variable: rxqueuesession? I have always wondered
about that variable).
I was able to get around the problems when the rxqueue class was
introduced. It allows you to create multiple instances of queues and
control them independently within the same instance of the
interpreter.
My example only showed the makearray method of the class. You will
need to read up on the other methods to apply this to your situation.
I find it easier to digest small bits of information at a time, so
that is usually the way I give it.
I say "instance of the interpreter" because, as Rick McGuire likes to
point out so often, "it's object all the way down".
Swifty
>> I do indeed. I was thinking the 'external data queue' doc was a bit
>> sparse on detail about precisely how this was prevented;
>
> a multithreaded environment.
And of course, you have precisely the same problem if you use
redirection into a file instead of queues - you have to pick a
system/thread wide unique filename.
Jeremy Nicoll - news posts
> WARNING! This info is based off of my usage of the interpreter and I
> have not reviewed all of the code in question. I'm sure Rick McGuire
> or Mark Miesfeld will jump in to correct me...
Clearly the documentation is very lacking in this respect.
It seems to me that queues are worth avoiding. Even if redirection into
files is not entirely safe, at least the ideas are much easier to grasp.
--
Jeremy C B Nicoll - my opinions are my own.
Email sent to my from-address will be deleted. Instead, please reply
to newsre...@wingsandbeaks.org.uk replacing "nnn" by "284".
Jeremy Nicoll - news posts
> And of course, you have precisely the same problem if you use redirection
> into a file instead of queues - you have to pick a system/thread wide
> unique filename.
For the duration of the execution of the command whose output is being
redirected, am I presuming too much about Windows to assume that the file is
not able to be used by any other process/thread/application on the system?
I do understand that after it's closed by the redirection ending it could be
written to or deleted by another process.
Nothing prevented that when I used VIO files the same way on MVS though they
were accessible only to other tasks in the same address space. And because
VIO files were created with an associated ddname but never cataloged, it
would need another subtask to use the same ddname to access the file.
Of course if anyone wrote apps that consistently allocated VIO files with a
fixed ddname, eg "VIOFILE7" that could happen (strictly a second allocation
would fail if the ddname was already in use, but a carelessly-written app
could ignore that and go on to use the already-allocated one).
I used to use a rexx function to give me a ddname that wasn't already in use
whenever I wanted to allocate such a file, so even multiple copies of the
same app running in the same address space could keep their temporary files
apart.
bwcherry
<jn.nntp.scrap...@wingsandbeaks.org.uk> wrote:
> bwcherry <brandonwche...@gmail.com> wrote:
> > WARNING! This info is based off of my usage of the interpreter and I
> > have not reviewed all of the code in question. I'm sure Rick McGuire
> > or Mark Miesfeld will jump in to correct me...
>
> Clearly the documentation is very lacking in this respect.
>
> It seems to me that queues are worth avoiding. Even if redirection into
> files is not entirely safe, at least the ideas are much easier to grasp.
>
> --
> Jeremy C B Nicoll - my opinions are my own.
>
> Email sent to my from-address will be deleted. Instead, please reply
I still think you need to read the RexxQueue class. It can take care
of the conflict in names if you understand how it works. Again, the
limits are still there for the BIFs (I assume for backwards
compatibility reasons) and older versions of the interpreter. This
class was first available in 3.2.0.
From the create method:
"Attempts to create an external Rexx named queue using name. If a name
queue already exists, a new queue with a Rexx-generated name will be
created. This method returns the name of the created queue, which will
be either name, or a generated name if there is a conflict."
If you feel the documentation needs updating, you should feel free to
submit any changes on the project page on sourceforge. There is a
tracker specifically for documentation updates.
--
Brandon Cherry
james laing
Rick McGuire
> Is there a way to access OORexx variables by name so that they can be bound by a third party application. I have a version of Pipelines that I want to be able incorporate into OORexx. I am trying to understand how I might implement the VAR stage into my version of Pipelines. 'www.tenfiftytwo.co.uk/pipelines'. Any pointers would be much appreciated.
>
>
variables. However, I doubt it will be much use to you. That API gives
you access to the variables running in the same process as the
interpreter. Since the pipelines stages run as a command that gets
launched in a new process, that API won't work.
Rick
ML
> variables.
> However, I doubt it will be much use to you.
Then use the RexxStart() API to launch a Rexx interpreter in the same
process, if that's required by the environment.
---