Using REXX to read html source

[Bill Ashton]

Hey everyone! Does anyone know if I can use TSO and REXX to issue a http:
URL and then read the output or source of the webpage for a page on our
company's intranet? If so, a little help to code this would be great!
Thanks!
--
Thank you and best regards,
*Billy Ashton*

----------------------------------------------------------------------
For TSO-REXX subscribe / signoff / archive access instructions,
send email to LIST...@VM.MARIST.EDU with the message: INFO TSO-REXX

[Paul Gilmartin]

On Jan 19, 2012, at 10:50, Bill Ashton wrote:

> Hey everyone! Does anyone know if I can use TSO and REXX to issue a http:
> URL and then read the output or source of the webpage for a page on our
> company's intranet? If so, a little help to code this would be great!
> Thanks!
>

Is RXSOCKET still around? I used it long ago, on TSO, and
more recently on CMS. But CMS/TSO pipelines, an endangered
species on TSO, makes it far easier.

See:

http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/f1a1d490/3.5

Title: z/OS V1R12.0 Comm Svr: IP Sockets Application Programming Interface Guide and Reference
Document Number: SC31-8788-09

3.5 Chapter 14. REXX socket application programming interface

I've done it; it's not pretty.

-- gil

[John McKown]

I'd use the UNIX curl command, in REXX via the "bpxwunix" command.
"curl" is part of the z/OS Ported Tools, 5655-M23.
<quote>
IBM Ported Tools for z/OS is a program product available at no cost to
you and is designed to deliver ported tools and utilities (modified to
operate within the z/OS environment) which are useful for application
development on, and secure communications to, the z/OS platform. IBM
Ported Tools for z/OS includes utilities such as OpenSSH, Perl, PHP,
bzip2, cURL, Xvfb, and IBM HTTP Server 7.0 where OpenSSH is in the
product base and the remaining items are separate features.
</quote>

Information on how to use curl is available with a Google search.
It's home page is here: http://curl.haxx.se/
What I got with z/OS 1.12 was curl 7.16.1
bpxunix() is a standard part of REXX in z/OS and is documented here:
http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/BPXZB6A0/5.2
bpxunix() is documented in a UNIX book, but works fine in TSO REXX too.

I do not have any example code.

--
John McKown
Maranatha! <><

[Lindy Mayfield]

Hi

This is something I started working on at some point. It retrieves a page and displays the source. Perhaps it may be a good start for you if you want to write in Rexx.

Regards,
Lindy Mayfield


/* Rexx */
/* A very simple Rexx program to "get" a web page */ Trace Off


/* Note that the parsing for this syntax wasn't done --
http://my.server:8080/page */

web_page = "http://your.server.com/your/webpage.html"
web_page = "http://www.sas.com/software/sas9/"

httpport = '80'
Verbose = 1
CRLF = '0D25'x

parse var web_page "http://"webserver"/"webfile

say "Getting file" webfile "from server" webserver

srv = Socket('Initialize', 'wget')
if verbose then
say 'Socket Initialized ===>' srv

srv = Socket('GetHostId')
parse var srv src ipaddress
if verbose then
say 'gethostid retd ===>' srv

srv= Socket('Socket')
parse var srv src ssockid
if verbose then
say 'socket retd ===>' srv
if src > 0 then signal bye

srv = Socket('Gethostbyname',webserver)
parse var srv src serverip
if verbose then
say 'gethbynm retd ===>' srv

srv = Socket('Setsockopt',ssockid,'SOL_SOCKET','SO_REUSEADDR','ON')
parse var srv src
if verbose then
say 'Setsockopt ===>' srv

srv = Socket('Setsockopt',ssockid,'SOL_SOCKET','SO_LINGER','OFF')
parse var srv src
if verbose then
say 'Setsockopt ===>' srv

srv = Socket('Setsockopt',ssockid,'SOL_SOCKET','SO_ASCII','ON')
parse var srv src
if verbose then
say 'Setsockopt ===>' srv

srv = Socket('Connect',ssockid,'AF_INET' httpport serverip) parse var srv src if verbose then
say 'Connect ===>' srv
If src > 0
then do
Say 'Connect to' webserver 'returned' srv
Say 'Error: Aborting process'
Return
End

Say ' '
Say 'Connected to server' webserver 'ip='serverip 'port='httpport Say ' '


/*
srv = Socket('Recv',ssockid)
parse var srv src len data

Say 'Server returned' len 'bytes of data:'
Say 'read>>' data
*/

say 'Sending HTTP request of' len 'bytes:'
httpget = 'GET' '/'webfile 'HTTP/1.0' CRLF srv = Socket('Send',ssockid,httpget) parse var srv src len say 'write>>' httpget

say 'Sending HTTP request of' len 'bytes:'
hosttxt = 'Host:' webserver':'httpport CRLF srv = Socket('Send',ssockid,hosttxt) parse var srv src len say 'write>>' hosttxt

/* Need one extra CRLF... */
srv = Socket('Send',ssockid,CRLF)
parse var srv src len

srv = Socket('Recv',ssockid)
parse var srv src len data

Say 'Server returned (headers)' len 'bytes of data:'
Say 'read>>' data

srv = Socket('Recv',ssockid)
parse var srv src len data

Say 'Server returned (data)' len 'bytes of data:'
Say 'read>>' data


bye:

srv = Socket('Close',ssockid)
if verbose then
say 'Close returned ===>' srv
Say ' '
say 'Closing Connection to server.'
Say ' '

return

[Bob Bridges]

What an EXTREMELY cool ability! Just one question: What's this SOCKET
thingy? Is it something that comes with MVS, or what? I don't see it in
the REXX manual.

---
Bob Bridges, rhb...@attglobal.net, cell 336 382-7313

/* I am a bomb technician. If you see me running, try and keep up! -a
T-shirt on a bomb squad member in Rio Rancho, NM */

-----Original Message-----
From: Lindy Mayfield
Sent: Saturday, January 21, 2012 10:56

This is something I started working on at some point. It retrieves a page
and displays the source. Perhaps it may be a good start for you if you want
to write in Rexx.

/* A very simple Rexx program to "get" a web page */

Say 'Connected to server' webserver 'ip='serverip 'port='httpport

say 'Sending HTTP request of' len 'bytes:'
httpget = 'GET' '/'webfile 'HTTP/1.0' CRLF
srv = Socket('Send',ssockid,httpget)
parse var srv src len
say 'write>>' httpget

say 'Sending HTTP request of' len 'bytes:'
hosttxt = 'Host:' webserver':'httpport CRLF
srv = Socket('Send',ssockid,hosttxt)
parse var srv src len say 'write>>' hosttxt

...

[John McKown]

On Sat, 2012-01-21 at 13:41 -0500, Bob Bridges wrote:
> What an EXTREMELY cool ability! Just one question: What's this SOCKET
> thingy? Is it something that comes with MVS, or what? I don't see it in
> the REXX manual.
>
> ---
> Bob Bridges, rhb...@attglobal.net, cell 336 382-7313

Not all REXX related abilities are in the "REXX" documentation area. As
another example, the use of REXX in and for ISPF is documented by the
ISPF team, not the REXX team, and is in the ISPF documentation area.
Likewise the use of REXX in and for TCPIP is written by the TCPIP team.
And is documented here:
http://publibz.boulder.ibm.com/cgi-bin/bookmgr_OS390/BOOKS/F1A1D490/3.5
It is a standard part of IBM's Communications Server. So, it "sort of"
comes with z/OS. Except that you could order z/OS without the
Communications Server feature. Of course, then you have no TCPIP or SNA
(VTAM) ability, but it is a possibility. Perhaps some very unusual shop
could do that by using CA-RUNTCP to do all the TCPIP stuff. That would
allow IP communications. Still no 3270 TSO, but perhaps they only run
z/OS UNIX shells? I get a headache thinking of it.


--
John McKown
Maranatha! <><

[Lindy Mayfield]

Yes, it is very cool. I was in pig heaven when I happened to run across this library of Rexx functions. There were though a few things that took me a bit of experimentation to figure out.

I actually wrote a simple web server to see if I could really do it, but it performed badly because it ran from TSO and was single threaded.

And I just saw John answered you where the Rexx TCP/IP docs are

(And speaking of John and TCP/IP, I spent a few hours trying to get GTK server to work from Rexx, but I gave up.)

Regards/Lindy

-----Original Message-----
From: TSO REXX Discussion List [mailto:TSO-...@VM.MARIST.EDU] On Behalf Of Bob Bridges
Sent: Saturday, January 21, 2012 8:41 PM
To: TSO-...@VM.MARIST.EDU
Subject: Re: [TSO-REXX] Using REXX to read html source

What an EXTREMELY cool ability! Just one question: What's this SOCKET thingy? Is it something that comes with MVS, or what? I don't see it in the REXX manual.

---
Bob Bridges, rhb...@attglobal.net, cell 336 382-7313

/* I am a bomb technician. If you see me running, try and keep up! -a T-shirt on a bomb squad member in Rio Rancho, NM */

[Bill Ashton]

Lindy, that works great, and gives me just what I need. Thanks so much for
your help!
Billy

--
Thank you and best regards,
*Billy Ashton*

[Bill Ashton]

Now that I have this working, I find that a single receive call is
truncating my data. Is there a maximum length for a string in Rexx?
Billy

[Paul Gilmartin]

On Jan 23, 2012, at 08:31, Bill Ashton wrote:

> Now that I have this working, I find that a single receive call is
> truncating my data. Is there a maximum length for a string in Rexx?
>

Is it possible that the code is exiting before receiving all packets?

But I've experienced a similar problem with RXSOCKET on VM/CMS. In
this case, the transfer would stall after a few megabytes. Timing
dependent: sometimes inserting a proxy would alleviate it. And
when my sysprog turned on some tracing, he was never able to
recreate it. At the moment, it's OK; I fear it will return.

-- gil

[Bill Ashton]

Gil, that's what appears to be happening. Is there a way to know how many
packets it takes, or is there some field I can check as I loop the GET
commands?
Billy

--
Thank you and best regards,
*Billy Ashton*

[Bob Bridges]

In VBA you can receive the HTML page as a document object; in that case you
can loop while waiting for a status code to indicate the whole thing has
been received. But I often find it useful to wait only until the existence
of a particular field I'm looking for.

That's no use in non-OO REXX/TSO, though. Here's an idea: I haven't gotten
into it yet (though you can be sure I'm going to!), but how about if you
confirm that the received string starts with <HTML> and then loop until
you've found </HTML>?

---
Bob Bridges, rhb...@attglobal.net, cell 336 382-7313

/* Because the future is profoundly unknowable, our best defense against
tyranny is often the courage to obey our vaguest premonitions -- or to
maintain what even the gentle and cheerful G.K. Chesterton called "a healthy
bigotry." -Joseph Sobran on conservatism */

-----Original Message-----
From: Bill Ashton
Sent: Monday, January 23, 2012 11:01

Gil, that's what appears to be happening. Is there a way to know how many
packets it takes, or is there some field I can check as I loop the GET
commands?

--- On Mon, Jan 23, 2012 at 10:52 AM, Paul Gilmartin
<PaulGB...@aim.com>wrote:

> Is it possible that the code is exiting before receiving all packets?
>
> But I've experienced a similar problem with RXSOCKET on VM/CMS. In
> this case, the transfer would stall after a few megabytes. Timing
> dependent: sometimes inserting a proxy would alleviate it. And
> when my sysprog turned on some tracing, he was never able to
> recreate it. At the moment, it's OK; I fear it will return.
>

> --- On Jan 23, 2012, at 08:31, Bill Ashton wrote:
> > Now that I have this working, I find that a single receive call is
> > truncating my data. Is there a maximum length for a string in Rexx?

[Steve Comstock]

On 1/23/2012 11:58 AM, Bob Bridges wrote:
> In VBA you can receive the HTML page as a document object; in that case you
> can loop while waiting for a status code to indicate the whole thing has
> been received. But I often find it useful to wait only until the existence
> of a particular field I'm looking for.
>
> That's no use in non-OO REXX/TSO, though. Here's an idea: I haven't gotten
> into it yet (though you can be sure I'm going to!), but how about if you
> confirm that the received string starts with<HTML> and then loop until
> you've found</HTML>?

Case-insensitive compare, of course.

--

Kind regards,

-Steve Comstock
The Trainer's Friend, Inc.

303-355-2752
http://www.trainersfriend.com

* To get a good Return on your Investment, first make an investment!
+ Training your people is an excellent investment

* Try our tool for calculating your Return On Investment
for training dollars at
http://www.trainersfriend.com/ROI/roi.html

[Bob Bridges]

Ok, I'm doing something I've never had to do before. I want to write a
utility that looks through all the members in a large PDS (2800-odd members,
I think) and pick out the ones that have more than a certain number of
records for extra processing. I figured this would call for me learn the LM
utilities. It took me a while to figure out how to hold my mouth right, but
now I suspect it's not going to tell me what I want after all: My test code
is giving me member names but no record counts. Here's what the relevant
part of my code looks like:

address ISPEXEC
'LMINIT DATAID(DID) DATASET('pds') ENQ(EXCLU)'
/* I use ENQ(EXCLU) because I may need to
update some of the members further down. */
'LMOPEN DATAID('did') OPTION(INPUT)'
mbr=''
do 12 /* just a test number */
'LMMLIST DATAID('did') MEMBER(MBR)'
if abbrev(mbr,'$') then iterate
'LMMFIND DATAID('did') MEMBER('mbr') STATS(YES)'
say mbr zlcnorc zlinorc zlmnorc; end
'LMMLIST DATAID('did') OPTION(FREE)'
'LMCLOSE DATAID(DID)'
'LMFREE DATAID('did')'

What I get back is member names and blanks. Am I to infer that this PDS
(which is created in batch) will never tell LMMFIND about the record counts
because ISPF hasn't saved any statistics for it? And if so... Well, if so,
what then? Must I read every member into the stack and check queued() to
see how big it is? Or can I find out the record size from some other
utility? I don't think LISTDS does it, for example.

---
Bob Bridges, rhb...@attglobal.net, cell 336 382-7313

/* Applying computer technology is simply a matter of finding the right
wrench to pound in the correct screw. */

[Lizette Koehler]

Being the LM commands are ISPF you might get a better response with the ISPF
newsgroup.


If you have not joined, you can go here and sigh up.

ISPF discussion https://listserv.nd.edu/cgi-bin/wa?A0=ISPF-L


Lizette

> -----Original Message-----
> From: TSO REXX Discussion List [mailto:TSO-...@VM.MARIST.EDU] On Behalf
Of
> Bob Bridges

[John McKown]

You are correct. If the member does not have ISPF statistics, then there
is NO WAY AT ALL to determine the number of records in the member except
by reading it. As an aside, the format of the ISPF statistics is
documented. So, it would be possible for an "evil user" to update them
with by writing a program which stores false information. I can't
imagine why anybody would do this.

z/OS itself does not keep a "record count" for a dataset in any place.
The closest is with VSAM and the record counts in the catalog. But those
are not guaranteed to be correct.

--
John McKown
Maranatha! <><

[Dave Salt]

When you look at the PDS using an ISPF member list, do you see any stats? If not then stats are off. If that's the case there are many other ways you could get the record count, one of which would be to call the ISPF VIEW service using the MACRO parameter. This would call a very simple macro that gets the last line number and passes it back to the calling REXX.

HTH,

Dave Salt

SimpList(tm) - try it; you'll get it!

http://www.mackinney.com/products/program-development/simplist.html



> Date: Sun, 5 Feb 2012 00:07:55 -0500
> From: star...@MINDSPRING.COM
> Subject: Re: LM facilities
> To: TSO-...@VM.MARIST.EDU

[Mickey]

I would use LM to pull the list a members and loop through it, but then I
would call an edit macro (you can use these with VIEW), and get the line
number of the last line using

Address 'ISPEXEC' 'ISREDIT MACRO'
Address "ISREDIT'
"(trash) = Linenum .ZLAST"
Address 'ISPEXEC" "VPUT (TRASH)"
"CANCEL"


--------------------------------------------------
From: "Bob Bridges" <rhb...@ATTGLOBAL.NET>
Sent: Saturday, February 04, 2012 11:07 PM

To: <TSO-...@VM.MARIST.EDU>
Subject: [TSO-REXX] LM facilities

[Walter Pachl]

Edit might not work for those (large ones) that he wants.

getcnt:
"ALLOC FI(IN) DA('pds("mem")') SHR REUSE"
cnt=0
Do Until rc<>0
'EXECIO 1000 DISKR IN (Stem L.'
cnt=cnt+L.0
End
'EXECIO 0 DISKR IN (FINIS'
'FREE FI(IN)'
Return cnt
UNTESTED
Walter
As an IBM Retiree I am happy about the microseconds burned :-)

[Binyamin Dissen]

On Sat, 4 Feb 2012 23:15:15 -0600 John McKown <joa...@SWBELL.NET> wrote:

:>You are correct. If the member does not have ISPF statistics, then there

:>is NO WAY AT ALL to determine the number of records in the member except
:>by reading it. As an aside, the format of the ISPF statistics is
:>documented. So, it would be possible for an "evil user" to update them
:>with by writing a program which stores false information. I can't
:>imagine why anybody would do this.

What can be done is sort by TTR and you can make a guess as to what is a
"large" member. Of course, "gas" may give false positives - but you can
eliminate the obvious small members.

:>z/OS itself does not keep a "record count" for a dataset in any place.

Binyamin Dissen <bdi...@dissensoftware.com>
http://www.dissensoftware.com

Director, Dissen Software, Bar & Grill - Israel


Should you use the mailblocks package and expect a response from me,
you should preauthorize the dissensoftware.com domain.

I very rarely bother responding to challenge/response systems,
especially those from irresponsible companies.

[Binyamin Dissen]

On Sun, 5 Feb 2012 10:33:29 +0100 Adrian Stern <adrian...@TELE2.SE> wrote:

:>Architecture, architecture, architecture!

:>You say the pds is created in batch? Then the program that creates the
:>members knows the number of lines in each. That simple!

Because every batch program not only counts the records, but even stores them
externally?

:>-----Original Message-----

:>From: TSO REXX Discussion List [mailto:TSO-...@VM.MARIST.EDU] On Behalf Of
:>Bob Bridges

:>Sent: den 5 februari 2012 05:08
:>To: TSO-...@VM.MARIST.EDU
:>Subject: [TSO-REXX] LM facilities

:>Ok, I'm doing something I've never had to do before. I want to write a

--
Binyamin Dissen <bdi...@dissensoftware.com>
http://www.dissensoftware.com

Director, Dissen Software, Bar & Grill - Israel


Should you use the mailblocks package and expect a response from me,
you should preauthorize the dissensoftware.com domain.

I very rarely bother responding to challenge/response systems,
especially those from irresponsible companies.

[John McKown]

I think that the respondent was saying that the batch program(s) which
write the member __COULD BE__ modified to count the records and maintain
ISPF member statistics. Of course, if the program is written in COBOL
and doing a sequential output to a member of a PDS specified in the JCL
DD statement, this is not a possibility.

Wouldn't that be a "neat" JCL and QSAM/BSAM/BPAM enhancement? New JCL
parameter which makes STOW maintain ISPF statistics. Yeah, like that's
going to happen. Way too many possible problems.

But, if ISPF stats were really critical, I could see writing to a
temporary sequential file. Then have the next step be an ftp step which
uses an ftp to LOCALHOST with do a "QUOTE SITE ISPFSTATS" before doing
the PUT.

--
John McKown
Maranatha! <><

[Vitonis, Tony]

Yep, every batch program would have to do that. What's more likely to get you a reliable record count?

- Relying on external programs, present and future, to store the count; then reading
a file or database to get that figure.
- Counting the records yourself.

Imagine that the writes are done in various places, and the read in only one. If the record count is maintained externally, potentially all of the writers would have to go through that overhead - just so a single reader can "know" how many records are in the file.

Architecture indeed.

-----Original Message-----
From: Binyamin Dissen

Because every batch program not only counts the records, but even stores them
externally?

-----Original Message-----
From: Adrian Stern

Architecture, architecture, architecture!

You say the pds is created in batch? Then the program that creates the members knows the number of lines in each. That simple!

-----Original Message-----
From: Bob Bridges

Ok, I'm doing something I've never had to do before. I want to write a utility that looks through all the members in a large PDS (2800-odd members, I think) and pick out the ones that have more than a certain number of records for extra processing. [snip]

[Vitonis, Tony]

That's exactly right: You have to look at the whole problem. Some problems aren't easily solved, or even solvable, in the way that you mention. Maybe I/O "should" be done in the way that you describe, but in existing products, sometimes it can't be done without major effort - effort whose result (a simple counting of records) isn't worth the development time.

The point is that you can't always rearchitect to take advantage of optimal solutions. In some cases, it's possible but not desirable. In this case, it's simply not that hard, and labor- or CPU-intensive, for the client to count the records in a file. Idealism is fine, but practical concerns will most often override it.

-----Original Message-----
From: TSO REXX Discussion List [mailto:TSO-...@VM.MARIST.EDU] On Behalf Of Adrian Stern
Sent: Sunday, February 05, 2012 2:52 PM
To: TSO-...@VM.MARIST.EDU

Subject: Re: LM facilities

Architecture, architecture, architecture!

If the object uses a single method for updating it then that method can keep
a record of the number of records written. This is how all i/o should be
done - through a single method irrespective of the process doing the
updating.

There is no extra overhead as far as I can see as the same number of mips
will be used - possibly fewer as the method may remain in memory and not
need to be fetched very often. - If you added a little overhead you could
use db2 instead of a pds and let it tell you how many of each.

Look at the whole problem - not just one aspect.

Adrian

-----Original Message-----
From: TSO REXX Discussion List [mailto:TSO-...@VM.MARIST.EDU] On Behalf Of

Vitonis, Tony
Sent: den 5 februari 2012 20:15
To: TSO-...@VM.MARIST.EDU
Subject: Re: [TSO-REXX] LM facilities

[Paul Gilmartin]

On Feb 5, 2012, at 12:52, Adrian Stern wrote:

> Architecture, architecture, architecture!
>
> If the object uses a single method for updating it then that method can keep
> a record of the number of records written. This is how all i/o should be
> done - through a single method irrespective of the process doing the
> updating.
>

That's called an "Access Method". That's where such
statistics should be kept, not by individual applications
such as ISPF, FTP, NFS, (others?), all of which maintain
and update member statistics, sometimes correctly; sometimes
incorrectly.

> There is no extra overhead as far as I can see as the same number of mips
> will be used - possibly fewer as the method may remain in memory and not
> need to be fetched very often. - If you added a little overhead you could
> use db2 instead of a pds and let it tell you how many of each.
>
> Look at the whole problem - not just one aspect.
>

Amen.

Does the OP require a precise record count, or only a rough
estimate? If the latter, he should consider converting from
PDS(E) to UNIX directories, which always maintain file sizes
in bytes. Unix files are supported by BSAM, QSAM, and half
of BPAM (read).

The pity is despite recurrent claims of better UNIX support in
successive releases of ISPF, I still get:

8 *-* 'LMINIT DATAID(LM) DDNAME('HFS')'
+++ RC(8) +++
ISRZ002 Data sets not on direct access device are not supported by ISPF.

WTF? What sort of devices are UNIX files on if not direct access?

Also a pity that Rexx provides no way (outside of LM) to access
multiple members of a single PDS without the overhead of ALLOCATE-
OPEN-CLOSE-FREE for each member.

Also a pity that ISREDIT provides no way to copy to/from
a Rexx numbered stem other than looping and incurring
the cost of an ISREDIT call for each line.

-- gil

[Bob Bridges]

Nah, they're not THAT big, just a few thousand lines, perhaps never more
than 10 000. I didn't give you more details before because I didn't think
them relevant, but as usual I was wrong: For those who are interested, my
client is loading a bunch of ACF2 dataset rules. SOME of those rules exceed
the installation's 32KB limit after compilation, and therefore the client
has created a REXX to break up those too-large rule sets into pieces of
(more or less arbitrarily) 4000 lines apiece. That logic is already created
to the client's satisfaction, but there's a problem with the sorting: ACF2
will sort the rules properly during compilation, but the ones that exceed
the 32K limit can't be compiled, so before breaking them up with NEXTKEYs we
must sort those rules ourselves. We have a sort sequence that should work;
my job is to write a REXX that goes through the 2800 members, picks out
those that exceed our limit, and re-sort only those members before feeding
the PDS to the next step.

There are some good ideas here, but I'm still reading...

---
Bob Bridges, rhb...@attglobal.net, cell 336 382-7313

/* If you are not being criticized, you may not be doing much. -Donald
Rumsfeld */

-----Original Message-----
From: Walter Pachl
Sent: Sunday, February 5, 2012 01:36

Edit might not work for those (large ones) that he wants.

----- Original Message -----
From: "Mickey" <mic...@COMCAST.NET>
Sent: Sunday, February 05, 2012 7:22 AM

>I would use LM to pull the list a members and loop through it, but then I
> would call an edit macro (you can use these with VIEW), and get the line
> number of the last line using
>
> Address 'ISPEXEC' 'ISREDIT MACRO'
> Address "ISREDIT'
> "(trash) = Linenum .ZLAST"
> Address 'ISPEXEC" "VPUT (TRASH)"
> "CANCEL"

[Bob Bridges]

Lisa suggested the Edit macro, too, but I'm assuming that it would take
longer to run even a quick Edit macro on 2800 members than it would to
simply do an EXECIO on each and check queued(). Comments?

I came up with another idea, but let me finish reading through all these
responses before I propose it.

---
Bob Bridges, rhb...@attglobal.net, cell 336 382-7313

/* Applying computer technology is simply a matter of finding the right
wrench to pound in the correct screw. */

-----Original Message-----
From: Mickey
Sent: Sunday, February 5, 2012 01:23

I would use LM to pull the list a members and loop through it, but then I
would call an edit macro (you can use these with VIEW), and get the line
number of the last line using

Address 'ISPEXEC' 'ISREDIT MACRO'
Address "ISREDIT'
"(trash) = Linenum .ZLAST"
Address 'ISPEXEC" "VPUT (TRASH)"
"CANCEL"

--------------------------------------------------
From: "Bob Bridges" <rhb...@ATTGLOBAL.NET>
Sent: Saturday, February 04, 2012 11:07 PM

[Paul Gilmartin]

On Feb 6, 2012, at 09:42, Bob Bridges wrote:

> Nah, they're not THAT big, just a few thousand lines, perhaps never more
> than 10 000. I didn't give you more details before because I didn't think
> them relevant, but as usual I was wrong: For those who are interested, my
> client is loading a bunch of ACF2 dataset rules. SOME of those rules exceed
> the installation's 32KB limit after compilation, and therefore the client
> has created a REXX to break up those too-large rule sets into pieces of
> (more or less arbitrarily) 4000 lines apiece. That logic is already created
> to the client's satisfaction, but there's a problem with the sorting: ACF2
> will sort the rules properly during compilation, but the ones that exceed
> the 32K limit can't be compiled, so before breaking them up with NEXTKEYs we
> must sort those rules ourselves. We have a sort sequence that should work;
> my job is to write a REXX that goes through the 2800 members, picks out
> those that exceed our limit, and re-sort only those members before feeding
> the PDS to the next step.
>
> There are some good ideas here, but I'm still reading...
>

I'm thinking, "EXECIO 4001 DISKR DDN (skip" and checking
for RC==0. Dammit; you'd need to ALLOCATE and FREE each
member. Go out for lunch while it runs. If only Rexx
would let you select a member without a separate ALLOCATE.
If only LM/ISREDIT facilities understood compound
variables. Ah! I RTFM. Something like that is there:
"LMGET MODE(MULTX)". But you get to unpack the data
yourself. A pathetic substitute for reading directly
into a compound variable.

-- gil

[Bob Bridges]

Doh! <Slap forehead> The logic that creates this PDS isn't mine, and I was
trying to post-process it without pestering the (assembler!) programmer who
generated it. But this is a simple enough change that I imagine he'd make
it if necessary, generating a side file that would list record count for
each member. Yeah, I may end up asking him to do that.

---
Bob Bridges, rhb...@attglobal.net, cell 336 382-7313

/* School that day consisted of a field trip to see a live performance of "A
Christmas Carol". This incensed my son, who believes cultural benefit can
only be found in entertainment that involves gruesome death. "I hate the
whole thing," he fumed when he found out about it. "I hate the ghost of
Christmas presents, I hate the guy who baas 'humbug,' and I hate Emily
Dickinson." -W Bruce Cameron, May 2002 */

-----Original Message-----
From: Adrian Stern
Sent: Sunday, February 5, 2012 04:33

Architecture, architecture, architecture!

You say the pds is created in batch? Then the program that creates the
members knows the number of lines in each. That simple!

-----Original Message-----
From: Bob Bridges

[Bob Bridges]

Ok, thanks for all this discussion (not excepting the digression against the
poor "Architecture, architecture!" guy, who after all made a suggestion that
was helpful in my situation). Although I may end up doing it his way, I did
come up with another possibility: It occurs to me that LISTDSI, while it
doesn't give me an exact record count, does give me allocated units and
percentage of allocation used. I should therefore be able to calculate that
72% of 5 tracks times 636 rec/track (at LRECL=88 - don't ask, just accept)
comes to 3663 records plus-or-minus 318. Thus if it tells me that a
particular member is....

Oh, wait a minute: I'll bet LISTDSI won't tell me how many tracks a member
has allocated, only the whole PDS. Right? Maybe I'll end up using Adrian's
idea after all. And if not, I can always fall back on doing 2800 EXECIOs.

---
Bob Bridges, rhb...@attglobal.net, cell 336 382-7313

/* Because the future is profoundly unknowable, our best defense against
tyranny is often the courage to obey our vaguest premonitions -- or to
maintain what even the gentle and cheerful G.K. Chesterton called "a healthy
bigotry." -Joseph Sobran on conservatism */

-----Original Message-----
From: TSO REXX Discussion List [mailto:TSO-...@VM.MARIST.EDU] On Behalf Of
Mickey
Sent: Sunday, February 5, 2012 01:23
To: TSO-...@VM.MARIST.EDU
Subject: Re: [TSO-REXX] LM facilities

[Paul Gilmartin]

On Feb 6, 2012, at 10:14, Bob Bridges wrote:
>
> Oh, wait a minute: I'll bet LISTDSI won't tell me how many tracks a member
> has allocated, only the whole PDS. Right? Maybe I'll end up using Adrian's
> idea after all. And if not, I can always fall back on doing 2800 EXECIOs.
>

EXECIO cheap; ALLOCATE expensive (relatively).

Y'know, PDSE likely knows how many records or pages are
allocated to each member. It probably won't confess.

-- gil

[Dave Salt]

Running a very simple edit macro (or in this particular case a 'view' macro that simply returns the line count) against 2,800 members would be extremely fast and efficient. If I were a betting man I'd say it would be MUCH faster than using EXECIO. This is especially true if VIEW DATAID is used instead of VIEW DATASET, where DATAID was previously established by the LMINIT.

There are many ways to execute a macro against every member in a PDS, with SimpList being one of them. If you have SimpList you can even set it so that any members that do or do not pass any test you want (such as member size or contents or name etc) could be excluded from the member list. For example, any members with less than a certain numbers of lines could be excluded, so that only larger members are left in the list. You could then enter 'C *' on the member list command line to copy all remaining members to another data set for further processing, or simply process them right there in the current member list. You could even enhance the macro to automatically do whatever it is that needs to be done, so that the entire process is completed in one fell swoop.

Dave Salt

SimpList(tm) - try it; you'll get it!

http://www.mackinney.com/products/program-development/simplist.html

> Date: Mon, 6 Feb 2012 11:42:44 -0500
> From: rhb...@ATTGLOBAL.NET

> Subject: Re: LM facilities
> To: TSO-...@VM.MARIST.EDU
>

[Nick Poulos]

Bob,

If this PDS is created in batch and does not have statistics (or only some members have statistics), you could set the statistics for the ones that do not have statistics with a single call to LMMSTATS before you do the LMMLIST.

I use this technique to get line counts for members that I have created using IEBUPDTE.

"ISPEXEC LMMSTATS DATAID("XDSID") MEMBER(*) "

Nick P. Poulos
Solutions Architect | Applications Management
nick....@asg.com
ASG-CloudFactory - The Single Solution to Build, Deploy & Manage your Cloud Environments  

-----Original Message-----
From: TSO REXX Discussion List [mailto:TSO-...@VM.MARIST.EDU] On Behalf Of Bob Bridges
Sent: Saturday, February 04, 2012 10:08 PM
To: TSO-...@VM.MARIST.EDU
Subject: [TSO-REXX] LM facilities

[Ted MacNEIL]

Look up the service LMSTATS.
I don't remember exactly, but I think it handles wild cards and can generate stats for members that don't have them, including line counts -- it's been a long time since I used it.

------Original Message------
From: Bob Bridges
Sender: TSO REXX Discussion List
To: TSO-...@VM.MARIST.EDU
ReplyTo: TSO REXX Discussion List
Subject: Re: [TSO-REXX] LM facilities

Sent: 6 Feb 2012 11:42

Lisa suggested the Edit macro, too, but I'm assuming that it would take
longer to run even a quick Edit macro on 2800 members than it would to
simply do an EXECIO on each and check queued(). Comments?

I came up with another idea, but let me finish reading through all these
responses before I propose it.

---
Bob Bridges, rhb...@attglobal.net, cell 336 382-7313

/* Applying computer technology is simply a matter of finding the right
wrench to pound in the correct screw. */

-----Original Message-----
From: Mickey
Sent: Sunday, February 5, 2012 01:23

I would use LM to pull the list a members and loop through it, but then I
would call an edit macro (you can use these with VIEW), and get the line
number of the last line using

Address 'ISPEXEC' 'ISREDIT MACRO'
Address "ISREDIT'
"(trash) = Linenum .ZLAST"
Address 'ISPEXEC" "VPUT (TRASH)"
"CANCEL"

--------------------------------------------------
From: "Bob Bridges" <rhb...@ATTGLOBAL.NET>

Sent: Saturday, February 04, 2012 11:07 PM

----------------------------------------------------------------------
For TSO-REXX subscribe / signoff / archive access instructions,
send email to LIST...@VM.MARIST.EDU with the message: INFO TSO-REXX

-

Ted MacNEIL
eama...@yahoo.ca
Twitter: @TedMacNEIL

[Kopischke, David G.]

Be careful you don't do this for an ENDEVOR managed PDS. ENDEVOR stores its own change management information in the ISPF statistics areas. Other products might too, but I only know of ENDEVOR.

-----Original Message-----
From: TSO REXX Discussion List [mailto:TSO-...@VM.MARIST.EDU] On Behalf Of Nick Poulos
Sent: Monday, February 06, 2012 11:49 AM
To: TSO-...@VM.MARIST.EDU
Subject: Re: [TSO-REXX] LM facilities

------------------------------------------------------------------------------
This e-mail transmission may contain information that is proprietary, privileged and/or confidential and is intended exclusively for the person(s) to whom it is addressed. Any use, copying, retention or disclosure by any person other than the intended recipient or the intended recipient's designees is strictly prohibited. If you are not the intended recipient or their designee, please notify the sender immediately by return e-mail and delete all copies. OppenheimerFunds may, at its sole discretion, monitor, review, retain and/or disclose the content of all email communications.
==============================================================================

[Bob Bridges]

I wondered whether there was a way to create stats all in one sweep like
that (or swoop, if you prefer), but never got around to asking. Maybe
that's the way to go, then.

...Although I did find out that the program that creates this PDS also
generates a member with line counts in it. The counts aren't correct; they
seem to be long by about 5%. I'm still finding out why. Still, the limit I
was working with is more-or-less arbitrary; I don't need to know EXACTLY how
many records are in each member. Still (again), if it doesn't take too long
to call LMMSTATS first, I should give it a try, if only so I'll know how to
use it next time.

---
Bob Bridges, rhb...@attglobal.net, cell 336 382-7313

/* In order to write for "The A-Team", you'd have to be a much better writer
than most of those who write the evening news at networks and local stations
- forget about shows like "Hill Street Blues" or "The Muppet Show", where
writing REALLY counts. -Linda Ellerbee, _And So It Goes_ */

-----Original Message-----
From: Nick Poulos
Sent: Monday, February 6, 2012 13:49

If this PDS is created in batch and does not have statistics (or only some
members have statistics), you could set the statistics for the ones that do
not have statistics with a single call to LMMSTATS before you do the
LMMLIST.

I use this technique to get line counts for members that I have created
using IEBUPDTE.

"ISPEXEC LMMSTATS DATAID("XDSID") MEMBER(*) "

[Robert Zenuk]

Well... I got curious and ran a benchmark... I did not do 2800 members, but here is the code and what I ended up with...

Drum roll please...

PDSIO1 - 44 members in OPSROZ.GEM.NEW in 0.019329 seconds
PDSIO2 - 44 members in OPSROZ.GEM.NEW in 0.669946 seconds
PDSIO3 - 44 members in OPSROZ.GEM.NEW in 1.286039 seconds
PDSIO4 - 44 members in OPSROZ.GEM.NEW in 0.274897 seconds

What they do:

PDSIO1 - Basic LMMLIST and print member - no lines - for comparison
PDSIO2 - LMMLIST and ALLOC/EXECIO/FREE print number of lines - 2ND Place
PDSIO3 - LMMLIST/VIEW/.ZLAST Edit Macro print number of lines - 3RD Place
PDSIO4 - LMMSTATS DELETE/LMMSTATS/LMMLIST print number of lines - WINNER

Here is the code so you can try it yourself with your own PDS.
Sorry about the line numbers, I lose the indentation if I don't do it that way...

000001 /* rexx - PDSIO1 - basic LMMLIST and print member */
000002 x = time('r')
000003 arg dsn pat
000004 if pat = '' then pat = '*'
000005 count = 0
000006 address ISPEXEC
000007 "LMINIT DATAID(PDS) DATASET('"dsn"') ENQ(SHR)"
000008 "LMOPEN DATAID("pds")"
000009 do forever
000010 "LMMLIST DATAID("pds") OPTION(LIST) STATS(NO)",
000011 "MEMBER(ZLMEMBER) PATTERN("pat")"
000012 if RC <> 0 then leave
000013 say zlmember
000014 count = count + 1
000015 end
000016 "LMMLIST DATAID("pds") OPTION(FREE)"
000017 "LMCLOSE DATAID("pds")"
000018 "LMFREE DATAID("pds")"
000019 say count 'members in' dsn 'in' time('e') 'seconds'

000001 /* rexx - PDSIO2 - LMMLIST and ALLOC/EXECIO/FREE */
000002 x = time('r')
000003 arg dsn pat
000004 if pat = '' then pat = '*'
000005 count = 0
000006 address ISPEXEC
000007 "LMINIT DATAID(PDS) DATASET('"dsn"') ENQ(SHR)"
000008 "LMOPEN DATAID("pds")"
000009 do forever
000010 "LMMLIST DATAID("pds") OPTION(LIST) STATS(NO)",
000011 "MEMBER(ZLMEMBER) PATTERN("pat")"
000012 if RC <> 0 then leave
000013 pdsmem = dsn'('strip(zlmember)')'
000014 address TSO "ALLOC F(MEM) DA('"pdsmem"') SHR REUSE"
000015 address TSO "EXECIO * DISKR MEM (STEM MEM. FINIS"
000016 address TSO "FREE F(MEM)"
000017 say left(zlmember,8) mem.0 'lines'
000018 count = count + 1
000019 end
000020 "LMMLIST DATAID("pds") OPTION(FREE)"
000021 "LMCLOSE DATAID("pds")"
000022 "LMFREE DATAID("pds")"
000023 say count 'members in' dsn 'in' time('e') 'seconds'

000001 /* rexx - PDSIO3 - LMMLIST/VIEW/.ZLAST */
000002 x = time('r')
000003 arg dsn pat
000004 if pat = '' then pat = '*'
000005 count = 0
000006 address ISPEXEC
000007 "LMINIT DATAID(PDS) DATASET('"dsn"') ENQ(SHR)"
000008 "LMOPEN DATAID("pds")"
000009 do forever
000010 "LMMLIST DATAID("pds") OPTION(LIST) STATS(NO)",
000011 "MEMBER(ZLMEMBER) PATTERN("pat")"
000012 if RC <> 0 then leave
000013 "VIEW DATAID("pds") MEMBER("strip(zlmember)") MACRO(@LAST)"
000014 count = count + 1
000015 end
000016 "LMMLIST DATAID("pds") OPTION(FREE)"
000017 "LMCLOSE DATAID("pds")"
000018 "LMFREE DATAID("pds")"
000019 say count 'members in' dsn 'in' time('e') 'seconds'

000001 /* rexx - @LAST edit macro */
000002 address ISREDIT
000003 "MACRO"
000004 "CURSOR = .ZLAST"
000005 "(LAST) = CURSOR"
000006 "(MEMBER) = MEMBER"
000007 say left(member,8) last 'lines'
000008 "END"
000009 exit(1)

000001 /* rexx - PDSIO4 - LMMSTATS DELETE/LMMSTATS/LMMLIST print member */
000002 x = time('r')
000003 arg dsn pat
000004 if pat = '' then pat = '*'
000005 count = 0
000006 address ISPEXEC
000007 "LMINIT DATAID(PDS) DATASET('"dsn"') ENQ(SHR)"
000008 "LMMSTATS DATAID("pds") MEMBER(*) DELETE"
000009 "LMMSTATS DATAID("pds") MEMBER(*)"
000010 "LMOPEN DATAID("pds")"
000011 do forever
000012 "LMMLIST DATAID("pds") OPTION(LIST) STATS(YES)",
000013 "MEMBER(ZLMEMBER) PATTERN("pat")"
000014 if RC <> 0 then leave
000015 say left(zlmember,8) right(zlcnorc,10)
000016 count = count + 1
000017 end
000018 "LMMLIST DATAID("pds") OPTION(FREE)"
000019 "LMCLOSE DATAID("pds")"
000020 "LMFREE DATAID("pds")"
000021 say count 'members in' dsn 'in' time('e') 'seconds'

JCL for all 4..

//jobcard
//***************************************************************
//* BATCH ISPF *
//***************************************************************
//MEMLIST PROC DSN=,MEM=*,NUM=
//PDSIO1 EXEC PGM=IKJEFT01,PARM='ISPSTART CMD(%PDSIO&NUM &DSN &MEM)'
//SYSEXEC DD DSN=your.EXEC,DISP=SHR
//ISPPLIB DD DSN=your.ISPPLIB,DISP=SHR
//ISPSLIB DD DSN=your.ISPSLIB,DISP=SHR
//ISPMLIB DD DSN=your.ISPMLIB,DISP=SHR
//ISPTLIB DD DDNAME=ISPTABL
// DD DSN=your.ISPTLIB,DISP=SHR
//ISPTABL DD DSN=your.ISPTLIB,DISP=SHR
//ISPTABL DD LIKE=your.ISPTLIB,UNIT=VIO,DSORG=PO
//ISPPROF DD LIKE=your.ISPTLIB,UNIT=VIO,DSORG=PO
//ISPLOG DD SYSOUT=*,RECFM=VA,LRECL=125
//SYSTSPRT DD SYSOUT=*
//SYSTSIN DD DUMMY
// PEND
//PDSIO1 EXEC MEMLIST,DSN=some.pds,MEM=*,NUM=1
//PDSIO2 EXEC MEMLIST,DSN=some.pds,MEM=*,NUM=2
//PDSIO3 EXEC MEMLIST,DSN=some.pds,MEM=*,NUM=3
//PDSIO4 EXEC MEMLIST,DSN=some.pds,MEM=*,NUM=4

Rob

-----Original Message-----
From: Ted MacNEIL <eama...@YAHOO.CA>
To: TSO-REXX <TSO-...@VM.MARIST.EDU>
Sent: Mon, Feb 6, 2012 12:23 pm
Subject: Re: LM facilities


Look up the service LMSTATS.

don't remember exactly, but I think it handles wild cards and can generate

tats for members that don't have them, including line counts -- it's been a

ong time since I used it.

-----Original Message------
rom: Bob Bridges
ender: TSO REXX Discussion List
o: TSO-...@VM.MARIST.EDU
eplyTo: TSO REXX Discussion List
ubject: Re: [TSO-REXX] LM facilities
ent: 6 Feb 2012 11:42
Lisa suggested the Edit macro, too, but I'm assuming that it would take
onger to run even a quick Edit macro on 2800 members than it would to
imply do an EXECIO on each and check queued(). Comments?
I came up with another idea, but let me finish reading through all these

esponses before I propose it.
---

ob Bridges, rhb...@attglobal.net, cell 336 382-7313
/* Applying computer technology is simply a matter of finding the right

rench to pound in the correct screw. */
-----Original Message-----
rom: Mickey
ent: Sunday, February 5, 2012 01:23
I would use LM to pull the list a members and loop through it, but then I
ould call an edit macro (you can use these with VIEW), and get the line
umber of the last line using
Address 'ISPEXEC' 'ISREDIT MACRO'
ddress "ISREDIT'
(trash) = Linenum .ZLAST"
ddress 'ISPEXEC" "VPUT (TRASH)"
CANCEL"
--------------------------------------------------

rom: "Bob Bridges" <rhb...@ATTGLOBAL.NET>

or TSO-REXX subscribe / signoff / archive access instructions,

end email to LIST...@VM.MARIST.EDU with the message: INFO TSO-REXX
-

ed MacNEIL
amac...@yahoo.ca
witter: @TedMacNEIL

[Glenn Knickerbocker]

On 2/6/2012 12:09 PM, Paul Gilmartin wrote:
> I'm thinking, "EXECIO 4001 DISKR DDN (skip" and checking
> for RC==0. Dammit; you'd need to ALLOCATE and FREE each
> member. Go out for lunch while it runs. If only Rexx
> would let you select a member without a separate ALLOCATE.

Got BatchPipes? "PIPE < dd=" will read a member from an allocated data set.

"ALLOCATE DS("dsn") DD(MEMSIZE) SHR"

"PIPE (end /) tso listds" dsn "members" ,
"| frlabel --MEMBERS--" || ,
"| drop 1" ,
"| tolabel --" || ,
"| mem: fanout" ,
"| pad 11" ,
"| size: juxtapose" ,
"| stem member." ,
"/ mem:" ,
"| specs 'callpipe < dd=memsize(' 1 w1 n '||count lines||*:' n" ,
"| pipcmd" ,
"| pad left 10" ,
"| size:"

ŹR

[Bob Bridges]

Remember I said I was wrong to suppose the details don't matter? Within the
last hour I realized I left out another detail: The sort required an
ALTSEQ. So the Edit macro won't fly; it'll have to be a call to DFSORT.

---
Bob Bridges, rhb...@attglobal.net, cell 336 382-7313

/* I don't want to achieve immortality through my work, I want to achieve it
through not dying. -Woody Allen */

[Paul Gilmartin]

On 2/6/2012 4:09 PM, Robert Zenuk wrote:
>
> PDSIO4 - LMMSTATS DELETE/LMMSTATS/LMMLIST print number of lines - WINNER

But beware -- this may overflow the directory size of a PDS. PDSE is nicer.

>
> Sorry about the line numbers, I lose the indentation if I don't do it that way...
>

What misbegotten application curses you with that behavior?

-- gil

[Bob Bridges]

I assumed it was his email client.

---
Bob Bridges, rhb...@attglobal.net, cell 336 382-7313

/* Leadership is a combination of strategy and character. If you must be
without one, be without the strategy. -Gen. H. Norman Schwarzkopf */

-----Original Message-----
From: Paul Gilmartin
Sent: Monday, February 6, 2012 19:15

What misbegotten application curses you with that behavior?

--- On 2/6/2012 4:09 PM, Robert Zenuk wrote:
> Sorry about the line numbers, I lose the indentation if I don't do it that
way...

[Paul Gilmartin]

On 2/6/2012 5:34 PM, Bob Bridges wrote:
> I assumed it was his email client.
>

Intrusive; I hate when they do that. I'm imagining one
of these days regressing to "mutt".

-- gil

[Mickey]

I have written edit macros that do FAR FAR FAR more than this, and running
in batch, it is often a challenge to get to SDSF before the job completes.

--------------------------------------------------
From: "Bob Bridges" <rhb...@ATTGLOBAL.NET>

Sent: Monday, February 06, 2012 11:42 AM

To: <TSO-...@VM.MARIST.EDU>
Subject: Re: [TSO-REXX] LM facilities

[Mickey]

But why change the stats? Maybe they are left blank for a reason. Also, one
needs ACF2 or RACF update on the files to create stats. Running an edit
macro, all one needs is permission to view the file. As a great man once
said "The more you over think the plumping, the easier it is to stop up the
drain".

--------------------------------------------------
From: "Bob Bridges" <rhb...@ATTGLOBAL.NET>

Sent: Monday, February 06, 2012 3:46 PM

To: <TSO-...@VM.MARIST.EDU>
Subject: Re: [TSO-REXX] LM facilities

[julian...@gmail.com]

Could you try reading the members using lmmfind and lmget. Allocating and opening one member at a time is quite an overhead. I have utilities scanning thru libraries like this all the time and performance is rarely an issue.

One of the whole points of the lm code is to be able to allocate and open the pds only once.

This thread seems to express an aversion to using lm utilities. Am I right? If so why?

Thanks
--
Sent from my Android phone with K-9 Mail. Please excuse my brevity.

_____________________________________________

_____________________________________________

or TSO-REXX subscribe / signoff / archive access instructions,
end email to LIST...@VM.MARIST.EDU with the message: INFO TSO-REXX
-
ed MacNEIL
amac...@yahoo.ca
witter: @TedMacNEIL

_____________________________________________

[Mickey]

I would use LM to open the PDS and get the list of member names, and then
use the edit macro I provided to get the last line number and pass it back.
I have run macros like this over thousands of large cobol programs, and if
it is donein batch, they run EXTREMELY quickly.

Just my 3 cents (adjusted for inflation)

--------------------------------------------------
From: "julian...@gmail.com" <julian...@GMAIL.COM>
Sent: Tuesday, February 07, 2012 9:25 AM
To: <TSO-...@VM.MARIST.EDU>

Subject: Re: [TSO-REXX] LM facilities

[Bob Bridges]

This question has generated a lot more attention and suggestions than I ever
expected, and I appreciate it. Thanks, all.

Mickey, I don't think the stats were left blank for a reason, they were left
blank simply because the PDS was generated in batch by a program that wasn't
written to interact with ISPF. And it isn't that I want to change them,
exactly, I'm just looking for the most efficient way of finding out how many
records are in each member; a lot of folks here seem to think LMMSTATS will
do it faster than my doing an EXECIO * on every member.

I'm going to need update access to the PDS in any case; once I know which
members exceed a certain size, my job is to re-sort them (the little ones
don't matter) and write them back to the same PDS. Right now they're under
the HLQ of a coworker working on the same project, so the plan is to create
a shared HLQ for the purpose of this task and start using it together:
He'll create the original data, and I'll post-process it.

---
Bob Bridges, rhb...@attglobal.net, cell 336 382-7313

/* I also attended Friday night's baseball game between the Marlins and the
San Francisco Giants at the stadium here. It was originally named
"Candlestick Park", but the sponsorship was subsequently sold to a computer
company, which renamed the stadium after itself: "Nerd Park." No,
seriously, it is now officially known as "The Stadium Formerly Known As
Candlestick Park", or, as the locals sometimes call it for short,
"Candlestick Park". -Dave Barry, 1997 */

-----Original Message-----
From: Mickey
Sent: Tuesday, February 7, 2012 08:34

But why change the stats? Maybe they are left blank for a reason. Also, one
needs ACF2 or RACF update on the files to create stats. Running an edit
macro, all one needs is permission to view the file. As a great man once
said "The more you over think the plumping, the easier it is to stop up the
drain".

--------------------------------------------------
From: "Bob Bridges" <rhb...@ATTGLOBAL.NET>
Sent: Monday, February 06, 2012 3:46 PM

> I wondered whether there was a way to create stats all in one sweep like
> that (or swoop, if you prefer), but never got around to asking. Maybe
> that's the way to go, then.
>
> ...Although I did find out that the program that creates this PDS also
> generates a member with line counts in it. The counts aren't correct;
> they seem to be long by about 5%. I'm still finding out why. Still, the
> limit I was working with is more-or-less arbitrary; I don't need to know
> EXACTLY how many records are in each member. Still (again), if it doesn't
> take too long to call LMMSTATS first, I should give it a try, if only so
> I'll know how to use it next time.
>

> -----Original Message-----
> From: Nick Poulos
> Sent: Monday, February 6, 2012 13:49
>
> If this PDS is created in batch and does not have statistics (or only some
> members have statistics), you could set the statistics for the ones that
> do not have statistics with a single call to LMMSTATS before you do the
> LMMLIST.
>
> I use this technique to get line counts for members that I have created
> using IEBUPDTE.
>
> "ISPEXEC LMMSTATS DATAID("XDSID") MEMBER(*) "
>
> -----Original Message-----
> From: Bob Bridges
> Sent: Saturday, February 04, 2012 10:08 PM
>
> Ok, I'm doing something I've never had to do before. I want to write a
> utility that looks through all the members in a large PDS (2800-odd
> members, I think) and pick out the ones that have more than a certain
> number of records for extra processing. I figured this would call for me
> learn the LM utilities. It took me a while to figure out how to hold my
> mouth right, but now I suspect it's not going to tell me what I want after

> all: My test code is giving me member names but no record counts....Am I

[Bob Bridges]

I assumed (again with the assuming) that while an alloc, execio and free
would take up more space than using the LM services, doing LMGET on every
record would slow down the process quite a bit; I didn't consider it
seriously. Now maybe you'll all tell me it's not bad at all.

I don't think you're imagining an aversion to LM services, Julian, but you
may be imagining it on the part of anyone but me; most of my respondents
seem to be in favor of it. I rarely use it, myself, and mostly I suspect
that's because I don't know much about it, which in turn is because
(surprise!) I rarely use it. I looked into it in this case because I could
see that 2800 allocates called for a better solution.

So my choices seem to be these:

1) LMMSTATS to generate ISPF statistics for all 2800 members; then LMMLIST
STATS(YES) to get the record count for each member.

2) Use VIEW DATAID(...) MEMBER(...) MACRO(EDMAC) to pull up each member; in
EDMAC, use .ZLAST to get the record count. This SOUNDS as though it would
take a long time to run, but most of you believe otherwise so I should try
it. Guess I can use VPUT to return the record count to the calling routine.

3) Modify the code that generates this PDS to create a record-count listing.
I asked my coworker about this and it turns out he's in favor of it; he
wants to reüse the logic in the future so he figures it may as well work
right.

4) Use the old-fashioned ALLOC, EXECIO and FREE for every member. This is
what I was trying to avoid; it would take only a few extra minutes, but I'd
still rather learn a new and better technique if it's out there.

---
Bob Bridges, rhb...@attglobal.net, cell 336 382-7313

/* Applying computer technology is simply a matter of finding the right

wrench to pound in the correct screw. */

-----Original Message-----
From: julian...@gmail.com
Sent: Tuesday, February 7, 2012 09:25

Could you try reading the members using lmmfind and lmget. Allocating and
opening one member at a time is quite an overhead. I have utilities scanning
thru libraries like this all the time and performance is rarely an issue.
One of the whole points of the lm code is to be able to allocate and open
the pds only once.

This thread seems to express an aversion to using lm utilities. Am I right?
If so why?

--- Robert Zenuk <robz...@AOL.COM> wrote:
Well... I got curious and ran a benchmark... I did not do 2800 members, but
here is the code and what I ended up with...

Drum roll please...

PDSIO1 - 44 members in OPSROZ.GEM.NEW in 0.019329 seconds
PDSIO2 - 44 members in OPSROZ.GEM.NEW in 0.669946 seconds
PDSIO3 - 44 members in OPSROZ.GEM.NEW in 1.286039 seconds
PDSIO4 - 44 members in OPSROZ.GEM.NEW in 0.274897 seconds

What they do:

PDSIO1 - Basic LMMLIST and print member - no lines - for comparison
PDSIO2 - LMMLIST and ALLOC/EXECIO/FREE print number of lines - 2ND Place
PDSIO3 - LMMLIST/VIEW/.ZLAST Edit Macro print number of lines - 3RD Place
PDSIO4 - LMMSTATS DELETE/LMMSTATS/LMMLIST print number of lines - WINNER

-----Original Message-----
From: Ted MacNEIL <eama...@YAHOO.CA>
Sent: Mon, Feb 6, 2012 12:23 pm

Look up the service LMSTATS. don't remember exactly, but I think it handles

wild cards and can generate stats for members that don't have them,
including line counts -- it's been a long time since I used it.

-----Original Message------
rom: Bob Bridges
ent: 6 Feb 2012 11:42

Lisa suggested the Edit macro, too, but I'm assuming that it would take
onger to run even a quick Edit macro on 2800 members than it would to
imply do an EXECIO on each and check queued(). Comments?

-----Original Message-----
rom: Mickey
ent: Sunday, February 5, 2012 01:23

I would use LM to pull the list a members and loop through it, but then I
ould call an edit macro (you can use these with VIEW), and get the line
umber of the last line

[Dave Salt]

IMO number 2 is your best bet.

Dave Salt

SimpList(tm) - try it; you'll get it!

http://www.mackinney.com/products/program-development/simplist.html

> Date: Tue, 7 Feb 2012 11:44:27 -0500

> From: rhb...@ATTGLOBAL.NET
> Subject: Re: LM facilities
> To: TSO-...@VM.MARIST.EDU
>

[Robert Zenuk]

While we can debate multiple ways of accomplishing the same thing, I tend to like to get a benchmark once all the ideas are on the table... Like many of you, I have used all the techniques at various times, but never benchmarked them against each other for a common purpose. Yes, I had some time on my hands (actually needed a break from what I was doing to clear my head). I added an LMMFIND/LMGET approach called PDSIO5.

All these tests are running in batch (same job as seen in the prior email).

and here are the new results with a larger dataset with larger members (50% over 2000 lines):

PDSIO1 - 1436 members in MY.EXEC.NEW in 0.132425 seconds
PDSIO2 - 1436 members in MY.EXEC.NEW in 21.419272 seconds
PDSIO3 - 1436 members in MY.EXEC.NEW in 53.657526 seconds
PDSIO4 - 1436 members in MY.EXEC.NEW in 30.104675 seconds
PDSIO5 - 1436 members in MY.EXEC.NEW in 31.281468 seconds

Here are the JOBLOG stats...

STEPNAME PROCSTEP PROGRAM RC EXCP CPU SRB CLOCK SERV
PDSIO1 PDSIO IKJEFT01 00 424 .09 .00 .54 4799
PDSIO2 PDSIO IKJEFT01 00 11900 6.77 .07 21.82 366K
PDSIO3 PDSIO IKJEFT01 00 40580 8.35 .23 54.11 473K
PDSIO4 PDSIO IKJEFT01 00 19935 1.10 .11 30.52 73507
PDSIO5 PDSIO IKJEFT01 00 6087 15.35 .04 31.64 815K

Review of what they do...

PDSIO1 - Basic LMMLIST and print member - no lines - for comparison

PDSIO2 - LMMLIST and ALLOC/EXECIO/FREE print number of lines - WINNER
PDSIO3 - LMMLIST/VIEW/.ZLAST Edit Macro print number of lines - 4TH Place
PDSIO4 - LMMSTATS DELETE/LMMSTATS/LMMLIST print number of lines - 2ND Place (by a nose)
PDSIO5 - LMMLIST and LMMFIND/LMGET print number of lines - 3RD place

Analysis:

Surprisingly PDSIO2 LMMLIST/ALLOC/EXECIO/FREE was the fastest by ~30%. This aproach seemd to have the best balance between CPU and I/O.
PDSIO5 LMMLIST/LMMFIND/LMGET was looking the worst in the smaller dataset trial, but got better with volume and became the preferred second choice over the LMMSTATS approach since that approach was invasive to the PDS directory and possibly impacted other users of the member stats (maybe not for the OP's project, but overall). However, the LMMFIND/LMGET approach consumed almost 3 times the CPU as the EXECIO approach, 15 times the LMMSTATS approach and almost 2 times the Edit Macro approach. On the flip side, this technique was the lowest EXCP approach
PDSIO3 was the Edit Macro approach and seemed to be the laggard, but was still respectable. If more processing was done other than extract the last line and print the member and line count, I suspect we might have seen an economies of scale... Interestingly, the Edit Macro approach was the highest EXCP user. It would be interesting to know what is different about his I/O technique that makes him such an I/O pig...


The PDSIO5 code:

000001 /* rexx - PDSIO5 - LMMLIST/LMMFIND/LMGET */

000002 x = time('r')
000003 arg dsn pat
000004 if pat = '' then pat = '*'
000005 count = 0
000006 address ISPEXEC
000007 "LMINIT DATAID(PDS) DATASET('"dsn"') ENQ(SHR)"
000008 "LMOPEN DATAID("pds")"
000009 do forever
000010 "LMMLIST DATAID("pds") OPTION(LIST) STATS(NO)",
000011 "MEMBER(ZLMEMBER) PATTERN("pat")"
000012 if RC <> 0 then leave

000013 count = count + 1
000014 "LMMFIND DATAID("pds") MEMBER("strip(zlmember)") LRECL(MRECL)"
000015 if RC = 0 then
000016 linecnt = 0
000017 do forever
000018 "LMGET DATAID("pds") MODE(INVAR) DATALOC(RECVAR) DATALEN(RECLEN)",
000019 "MAXLEN("mrecl")"
000020 if RC = 8 then
000021 leave
000022 else
000023 linecnt = linecnt + 1
000024 end
000025 say left(zlmember,8) linecnt 'lines'
000026 end
000027 "LMMLIST DATAID("pds") OPTION(FREE)"
000028 "LMCLOSE DATAID("pds")"
000029 "LMFREE DATAID("pds")"
000030 say count 'members in' dsn 'in' time('e') 'seconds'


Now, it's back to the salt mines...

And YES, my email client sucks. AOL Web interface... That being said, I have reasons I stay with AOL (occasional contacts with consulting clients that ultimately generate revenue).


Rob

-----Original Message-----
From: Mickey <mic...@COMCAST.NET>
To: TSO-REXX <TSO-...@VM.MARIST.EDU>
Sent: Tue, Feb 7, 2012 8:19 am
Subject: Re: LM facilities


I would use LM to open the PDS and get the list of member names, and then

se the edit macro I provided to get the last line number and pass it back.

have run macros like this over thousands of large cobol programs, and if

t is donein batch, they run EXTREMELY quickly.
Just my 3 cents (adjusted for inflation)
--------------------------------------------------

rom: "julian...@gmail.com" <julian...@GMAIL.COM>

ent: Tuesday, February 07, 2012 9:25 AM

o: <TSO-...@VM.MARIST.EDU>

[Farley, Peter x23353]

PMFJI here, but VIEW (and EDIT too) will FAIL if the member is too big for the TSO user's region size to support in-memory VIEW. This is very common when TSO users are limited in the allowed REGION they may specify at LOGON. The VIEW will be turned into a BROWSE which has no macro capabilities, AFAIK.

If #3 was not feasible for political reasons, I would try #1 against #4 and use the more efficient of the two.

HTH

Peter

This message and any attachments are intended only for the use of the addressee and may contain information that is privileged and confidential. If the reader of the message is not the intended recipient or an authorized representative of the intended recipient, you are hereby notified that any dissemination of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by e-mail and delete the message and any attachments from your system.

[Jeff Byrum]

Did you see Robert Zenuk's actual test run results from yesterday?

Using an EDIT MACRO was the worst performer by a factor of 4 times over using LMMSTATS.

When you invoke VIEW, I believe ISPF makes a temporary copy of the member, same as if you used EDIT. So you've got the extra overhead of writing every line of each member, in addition to the necessity of reading every line.

If you really don't want to change the STATS, I'm betting that even just reading (LMGET) and counting all the lines in each member of the PDS within the LMINIT...LMFREE loop wouldn't be a lot worse than issuing LMMSTATS. It's a question of how LMMSTATS determines the number of lines in the member -- either he's got to read the whole member and count lines, or have some other magical way of calculating the *exact* number of lines (and what if the RECFM is V or VB?)

Of course all this is irrelevant if the PDS is small to moderate size *and* your cost per unit of work is low. If the PDS is *huge* and the cost per unit of work is high, a 400% difference in time could be a real killer.

Jeff

-----Original Message-----
From: TSO REXX Discussion List [mailto:TSO-...@VM.MARIST.EDU] On Behalf Of Dave Salt
Sent: Tuesday, February 07, 2012 11:59 AM

[Dave Salt]

Rob,

Given that EXECIO requires an ALLOC and FREE on every member I'm really surprised the view macro takes longer. I don't know if it would make much difference but there's no need for the macro to move the cursor to the last line of the member. Also, I suspect (but don't know) that ISPF might do more processing to END from view than what it would do for CANCEL. So, any chance you could repeat the test with the following macro?

/* rexx - @LAST edit macro */

address ISREDIT
"MACRO"
"LAST = LINENUM .ZLAST"
"(MEMBER) = MEMBER"
say left(member,8) last 'lines'
"CANCEL"
exit(1)

Cheers,

Dave Salt

SimpList(tm) - try it; you'll get it!

http://www.mackinney.com/products/program-development/simplist.html

> Date: Tue, 7 Feb 2012 12:11:14 -0500
> From: robz...@AOL.COM

> Subject: Re: LM facilities
> To: TSO-...@VM.MARIST.EDU
>

[Bob Bridges]

Ok, next phase. I thought I knew how to do this - heck, I WAS doing it! -
but now it isn't working and I can't figure out why.

Years ago I wrote a REXX sort subroutine; the calling program could put the
//SORTIN data on the stack, push the SYSIN card(s) on top, then call SORTQ
and get the //SORTOUT records back in the stack. It took me a while to
figure out how to get it right, but it worked, and I've used it occasionally
since.

...But not recently. Now I'm trying to do it again, and I had to change my
routine from SYNCSORT to DFSORT (dunno whether that's relevant), and somehow
SORT isn't seeing my SYSIN records. Can anyone figure out what I'm doing
wrong?

The boiled-down subroutine currently looks like this:

/* SYSIN is probably being used already. */
v1=msg('OFF')
'FREE DDN(SYSIN SORTIN SORTOUT SORTMSG SYSOUT)'
call msg v1

/* Allocate the DDs. */
attr='DSORG(PS) RECFM(F B) LRECL(88)'
trk=trunc(queued()/636)+1
'ALLOC DDN(SYSIN) SPACE(1) TRACK RECFM(F B) LRECL(80)'
'ALLOC DDN(SORTIN) SPACE('trk') TRACK' attr
'ALLOC DDN(SORTOUT) SPACE('trk') TRACK' attr
'ALLOC DDN(SYSOUT) SPACE(2) TRACK RECFM(V B) LRECL(259)'

/* Populate the input data. */
pull nsysin /* the number of //SYSIN records */
'EXECIO' nsysin 'DISKW SYSIN (FINIS'
'EXECIO' queued() 'DISKW SORTIN (FINIS'

address LINK 'SORT'
'EXECIO * DISKR SORTOUT (FINIS'
say queued() 'records sorted in the stack'
'FREE DDN(SYSIN SORTIN SORTOUT SORTMSG SYSOUT)'

I'm getting 0 records back in SORTOUT and a 16 RC from SORT, so I check
SYSOUT and see this:

1ICE000I 0 - CONTROL STATEMENTS FOR 5694-A01, Z/OS DFSORT V1R10 - 12:52 ON
TUE FEB 07, 2012 -
0 R " 0 " 0 " 0 " 0 " 0 " 0 " 0 " 0
R
ICE007A 1 SYNTAX ERROR
ICE010A 0 NO SORT OR MERGE CONTROL STATEMENT
ICE751I 0 C5-K90025 C6-K90025 E7-K62201
ICE052I 3 END OF DFSORT

I see that the messages are all ICE rather than DFS, but why? Does this
have to do with some crucial difference between SYNCSORT and DFSORT, or have
I without realizing it changed something that worked before, or what?

---
Bob Bridges, rhb...@attglobal.net, cell 336 382-7313

/* Religion is the last stronghold of freedom. When the state forces the
church to surrender, its victory will be complete. Of course it will insist
that it respects "the separation of church and state" -- as defined by the
state....If the state can define golf and Scouting, why should it leave
defining Catholicism and Judaism to priests and rabbis? This isn't a
prophecy. It's a simple extrapolation from experience, and we'll have no
right to be surprised when it comes to pass. -Joseph Sobran, 2001-07-05 */

[Bob Bridges]

Sorry, I should have mentioned that all my ALLOCs are coming back with RC 0,
and I'm able to View SYSIN and see what I put in there, so it isn't simply a
matter of the ALLOC failing for some reason. Of course, it may still have
to do with a DCB that SORT isn't willing to deal with, or something.

---
Bob Bridges, rhb...@attglobal.net, cell 336 382-7313

/* A University without students is like an ointment without a fly. -Ed
Nather, professor of astronomy at UT-Austin */

[Bob Bridges]

No problem with the region size; none of these members approach even 10K
records. I think the largest is just over 6K records.

---
Bob Bridges, rhb...@attglobal.net, cell 336 382-7313

/* Ever since Freud, we've been taught to look "deep" for the causes of
crime and misbehavior -- in early childhood, in repressed memories, in
unconscious "roots" of conduct. Psychology now enjoys the prestige
astrology once commanded in royal courts. -Joseph Sobran */

-----Original Message-----
From: Farley, Peter x23353
Sent: Tuesday, February 7, 2012 12:17

PMFJI here, but VIEW (and EDIT too) will FAIL if the member is too big for
the TSO user's region size to support in-memory VIEW. This is very common
when TSO users are limited in the allowed REGION they may specify at LOGON.
The VIEW will be turned into a BROWSE which has no macro capabilities,
AFAIK.

If #3 was not feasible for political reasons, I would try #1 against #4 and
use the more efficient of the two.

> -----Original Message-----
> From: Dave Salt
> Sent: Tuesday, February 07, 2012 11:59 AM
>

> IMO number 2 is your best bet.
>

> -----Original Message-----
> > Date: Tue, 7 Feb 2012 11:44:27 -0500
> > From: rhb...@ATTGLOBAL.NET
> >

> > -----Original Message-----
> > From: julian...@gmail.com
> > Sent: Tuesday, February 7, 2012 09:25
> >
> > Could you try reading the members using lmmfind and lmget. Allocating
> > and opening one member at a time is quite an overhead. I have utilities
> > scanning thru libraries like this all the time and performance is rarely
> > an issue. One of the whole points of the lm code is to be able to
> > allocate and open the pds only once.
> >
> > This thread seems to express an aversion to using lm utilities. Am I
> > right? If so why?

[Robert Zenuk]

I did change it to a CANCEL for the last test. I will use your version of the macro and try again... I have a meeting, so it will be later today.

Rob

-----Original Message-----
From: Dave Salt <ds...@HOTMAIL.COM>
To: TSO-REXX <TSO-...@VM.MARIST.EDU>
Sent: Tue, Feb 7, 2012 11:04 am
Subject: Re: LM facilities

Rob,
Given that EXECIO requires an ALLOC and FREE on every member I'm really

urprised the view macro takes longer. I don't know if it would make much

ifference but there's no need for the macro to move the cursor to the last line

f the member. Also, I suspect (but don't know) that ISPF might do more

rocessing to END from view than what it would do for CANCEL. So, any chance you

ould repeat the test with the following macro?
/* rexx - @LAST edit macro */

ddress ISREDIT
MACRO"
LAST = LINENUM .ZLAST"
(MEMBER) = MEMBER"

ay left(member,8) last 'lines'
CANCEL"
xit(1)
Cheers,

ave Salt
SimpList(tm) - try it; you'll get it!
http://www.mackinney.com/products/program-development/simplist.html


> Date: Tue, 7 Feb 2012 12:11:14 -0500
From: robz...@AOL.COM
Subject: Re: LM facilities
To: TSO-...@VM.MARIST.EDU

While we can debate multiple ways of accomplishing the same thing, I tend to

ike to get a benchmark once all the ideas are on the table... Like many of

ou, I have used all the techniques at various times, but never benchmarked them

gainst each other for a common purpose. Yes, I had some time on my hands

actually needed a break from what I was doing to clear my head). I added an

MMFIND/LMGET approach called PDSIO5.

All these tests are running in batch (same job as seen in the prior email).

and here are the new results with a larger dataset with larger members (50%

ver 2000 lines):

PDSIO1 - 1436 members in MY.EXEC.NEW in 0.132425 seconds
PDSIO2 - 1436 members in MY.EXEC.NEW in 21.419272 seconds
PDSIO3 - 1436 members in MY.EXEC.NEW in 53.657526 seconds
PDSIO4 - 1436 members in MY.EXEC.NEW in 30.104675 seconds
PDSIO5 - 1436 members in MY.EXEC.NEW in 31.281468 seconds

Here are the JOBLOG stats...

STEPNAME PROCSTEP PROGRAM RC EXCP CPU SRB CLOCK

ERV
PDSIO1 PDSIO IKJEFT01 00 424 .09 .00 .54

799
PDSIO2 PDSIO IKJEFT01 00 11900 6.77 .07 21.82

66K
PDSIO3 PDSIO IKJEFT01 00 40580 8.35 .23 54.11

73K
PDSIO4 PDSIO IKJEFT01 00 19935 1.10 .11 30.52

3507
PDSIO5 PDSIO IKJEFT01 00 6087 15.35 .04 31.64

15K

Review of what they do...

PDSIO1 - Basic LMMLIST and print member - no lines - for comparison
PDSIO2 - LMMLIST and ALLOC/EXECIO/FREE print number of lines - WINNER
PDSIO3 - LMMLIST/VIEW/.ZLAST Edit Macro print number of lines - 4TH Place
PDSIO4 - LMMSTATS DELETE/LMMSTATS/LMMLIST print number of lines - 2ND Place

by a nose)
PDSIO5 - LMMLIST and LMMFIND/LMGET print number of lines - 3RD place

Analysis:

Surprisingly PDSIO2 LMMLIST/ALLOC/EXECIO/FREE was the fastest by ~30%. This

proach seemd to have the best balance between CPU and I/O.
PDSIO5 LMMLIST/LMMFIND/LMGET was looking the worst in the smaller dataset

rial, but got better with volume and became the preferred second choice over

he LMMSTATS approach since that approach was invasive to the PDS directory and

ossibly impacted other users of the member stats (maybe not for the OP's

roject, but overall). However, the LMMFIND/LMGET approach consumed almost 3

imes the CPU as the EXECIO approach, 15 times the LMMSTATS approach and almost

times the Edit Macro approach. On the flip side, this technique was the

owest EXCP approach
PDSIO3 was the Edit Macro approach and seemed to be the laggard, but was still

espectable. If more processing was done other than extract the last line and

rint the member and line count, I suspect we might have seen an economies of

cale... Interestingly, the Edit Macro approach was the highest EXCP user. It

ould be interesting to know what is different about his I/O technique that

akes him such an I/O pig...


The PDSIO5 code:

000001 /* rexx - PDSIO5 - LMMLIST/LMMFIND/LMGET */
> 000002 x = time('r')
> 000003 arg dsn pat
> 000004 if pat = '' then pat = '*'
> 000005 count = 0
> 000006 address ISPEXEC
> 000007 "LMINIT DATAID(PDS) DATASET('"dsn"') ENQ(SHR)"
> 000008 "LMOPEN DATAID("pds")"
> 000009 do forever
> 000010 "LMMLIST DATAID("pds") OPTION(LIST) STATS(NO)",
> 000011 "MEMBER(ZLMEMBER) PATTERN("pat")"
> 000012 if RC <> 0 then leave
> 000013 count = count + 1
> 000014 "LMMFIND DATAID("pds") MEMBER("strip(zlmember)") LRECL(MRECL)"
> 000015 if RC = 0 then
> 000016 linecnt = 0
> 000017 do forever
> 000018 "LMGET DATAID("pds") MODE(INVAR) DATALOC(RECVAR)

ATALEN(RECLEN)",
000019 "MAXLEN("mrecl")"
> 000020 if RC = 8 then
> 000021 leave
> 000022 else
> 000023 linecnt = linecnt + 1
> 000024 end
> 000025 say left(zlmember,8) linecnt 'lines'
> 000026 end
> 000027 "LMMLIST DATAID("pds") OPTION(FREE)"
> 000028 "LMCLOSE DATAID("pds")"
> 000029 "LMFREE DATAID("pds")"
> 000030 say count 'members in' dsn 'in' time('e') 'seconds'
>

Now, it's back to the salt mines...

And YES, my email client sucks. AOL Web interface... That being said, I have

easons I stay with AOL (occasional contacts with consulting clients that

[Binyamin Dissen]

It appears that you did not populate SYSIN.

Before the LINK do:

"EXECIO * DISKR SYSIN (FINIS STEM SYSIN."
say SYSIN.0 "SYSIN lines"
do ii = 1 to SYSIN.0
say "SYSIN line" ii "'"SYSIN.ii"'"
end

On Tue, 7 Feb 2012 13:15:36 -0500 Bob Bridges <rhb...@ATTGLOBAL.NET> wrote:

:>Ok, next phase. I thought I knew how to do this - heck, I WAS doing it! -

--
Binyamin Dissen <bdi...@dissensoftware.com>
http://www.dissensoftware.com

Director, Dissen Software, Bar & Grill - Israel


Should you use the mailblocks package and expect a response from me,
you should preauthorize the dissensoftware.com domain.

I very rarely bother responding to challenge/response systems,
especially those from irresponsible companies.

[Bob Bridges]

Oh, and by the way, it works in batch JCL; the problem, as far as I can
tell, has to be in the way I'm coding it on-line.

[Later:] Wait, hold everything: I just changed "address LINK 'SORT' " to
"address LINKMVS 'SORT' " and it worked! What's the difference?

---
Bob Bridges, rhb...@attglobal.net, cell 336 382-7313

/* Sure, I make my password my pet's name. My macaw's name is W47qY!3k, but
I change it every 90 days. */

-----Original Message-----
From: Bob Bridges [mailto:rhb...@attglobal.net]
Sent: Tuesday, February 7, 2012 13:31

Sorry, I should have mentioned that all my ALLOCs are coming back with RC 0,
and I'm able to View SYSIN and see what I put in there, so it isn't simply a
matter of the ALLOC failing for some reason. Of course, it may still have
to do with a DCB that SORT isn't willing to deal with, or something.

-----Original Message-----
From: Bob Bridges [mailto:rhb...@attglobal.net]

Sent: Tuesday, February 7, 2012 13:16

Ok, next phase. I thought I knew how to do this - heck, I WAS doing it! -
but now it isn't working and I can't figure out why.

Years ago I wrote a REXX sort subroutine; the calling program could put the
//SORTIN data on the stack, push the SYSIN card(s) on top, then call SORTQ

and get the //SORTOUT records back in the stack. It took me a while to

[Karl-Heinz Wittemann]

Anyway you have to write something like SORT
FIELDS=(startpos,length,format,order...) to SYSIN in fron you make a call
to sort.

Zitat von Bob Bridges <rhb...@ATTGLOBAL.NET>:

> instructions,send email to LIST...@VM.MARIST.EDU with the message:
> INFO TSO-REXX

/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/
Fon:   +49(0)7083 525732
Fax:   +49(0)7083 526877
Mobil: +49(0)176 562 44 567

Privat:+49(0)7083 524965
/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/

[julian...@gmail.com]

Good to get some benchmarks. I do use edit macros in batch at times, but I decide which technique to use based on requirements (or how I can imagine the solution). Performance is rarely critical and when it is you can profile/benchmark.

I stopped using alloc foreach member some time ago. I had code that took ages using alloc and execio, I changed it to use lm utils and the code flew. I do not remember the details and it could be some things have changed on zos in the intervening 10+ years!

One thing has certainly changed and thats the storage available.
--
Julian Levens

Sent from my Android phone with K-9 Mail. Please excuse my brevity.

[Paul Gilmartin]

On 2/7/2012 12:09 PM, Bob Bridges wrote:
>
> [Later:] Wait, hold everything: I just changed "address LINK 'SORT' " to
> "address LINKMVS 'SORT' " and it worked! What's the difference?
>

For nostalgic reasons, LINK uses a CMS calling sequence; LINKMVS
uses an MVS calling sequence . All in the book. LINKPGM, like
assembler CALL lets you use the first two bytes of parameters for
purposes other than string length.


>
> ---

>
> /* Populate the input data. */
> pull nsysin /* the number of //SYSIN records */
> 'EXECIO' nsysin 'DISKW SYSIN (FINIS'
> 'EXECIO' queued() 'DISKW SORTIN (FINIS'
>

The stack scares me -- it's way easy to get out of sync. I rely
on stem variables and/or 1-record EXECIO instead.

-- gil

[Glenn Knickerbocker]

On 2/6/2012 6:10 PM, Robert Zenuk wrote:
> 000008 "LMMSTATS DATAID("pds") MEMBER(*) DELETE"
> 000009 "LMMSTATS DATAID("pds") MEMBER(*)"

I checked Time('E') after each of these and was astounded to see that
the DELETE took just as long as generating the new statistics! So if
you can trust any existing stats, you can speed this up by double and
more by skipping the DELETE.

Even when the stats are already there, LMMSTATS seems to have to read
the whole data set. In my trials, it takes between 60% and 95% of the
time of my Pipelines solution. When it's generating stats, it runs 2-3
times as long as my Pipelines solution.

ŹR

[Don Leahy]

A low tech way to determine member sizes is to use SRCHFOR with the LNFMTO
option. If you search for a string that is unlikely to exist in any of the
members (x'FF', for example), LMTO will create a report listing each member
along with the number of records searched in each. From there all you have
to do is parse the report to identify the large members.

On Tue, Feb 7, 2012 at 16:22, Glenn Knickerbocker <No...@bestweb.net> wrote:

> On 2/6/2012 6:10 PM, Robert Zenuk wrote:
> > 000008 "LMMSTATS DATAID("pds") MEMBER(*) DELETE"
> > 000009 "LMMSTATS DATAID("pds") MEMBER(*)"
>
> I checked Time('E') after each of these and was astounded to see that
> the DELETE took just as long as generating the new statistics! So if
> you can trust any existing stats, you can speed this up by double and
> more by skipping the DELETE.
>
> Even when the stats are already there, LMMSTATS seems to have to read
> the whole data set. In my trials, it takes between 60% and 95% of the
> time of my Pipelines solution. When it's generating stats, it runs 2-3
> times as long as my Pipelines solution.
>

> ¬R

[Robert Zenuk]

It did not make much difference...

PDSIO1 - 1436 members in OPSROZ.EXEC.NEW in 0.104609 seconds
PDSIO2 - 1436 members in OPSROZ.EXEC.NEW in 17.862877 seconds
PDSIO3 - 1436 members in OPSROZ.EXEC.NEW in 44.883722 seconds
PDSIO4 - 1436 members in OPSROZ.EXEC.NEW in 32.393028 seconds
PDSIO5 - 1436 members in OPSROZ.EXEC.NEW in 23.156015 seconds

However, in this run PDSIO5 (LMMFIND/LMGET) was faster than PDSIO4 (LMMSTATS) and closed the gap with PDSIO2 (ALLOC/EXECIO)

#STEPNAME PROCSTEP PROGRAM RC EXCP CPU SRB CLOCK SERV
#PDSIO1 PDSIO IKJEFT01 00 439 .07 .00 .57 4185
#PDSIO2 PDSIO IKJEFT01 00 12137 6.02 .05 18.29 326K
#PDSIO3 PDSIO IKJEFT01 00 40692 7.32 .19 45.31 416K
#PDSIO4 PDSIO IKJEFT01 00 21202 .98 .09 32.81 66716
#PDSIO5 PDSIO IKJEFT01 00 6084 14.27 .04 23.55 758K

Rob

-----Original Message-----
From: Dave Salt <ds...@HOTMAIL.COM>

To: TSO-REXX <TSO-...@VM.MARIST.EDU>
Sent: Tue, Feb 7, 2012 11:04 am
Subject: Re: LM facilities

Rob,
Given that EXECIO requires an ALLOC and FREE on every member I'm really

urprised the view macro takes longer. I don't know if it would make much

ifference but there's no need for the macro to move the cursor to the last line

f the member. Also, I suspect (but don't know) that ISPF might do more

rocessing to END from view than what it would do for CANCEL. So, any chance you

ould repeat the test with the following macro?
/* rexx - @LAST edit macro */

ddress ISREDIT
MACRO"
(LAST) = LINENUM .ZLAST"
(MEMBER) = MEMBER"

ay left(member,8) last 'lines'
CANCEL"
xit(1)
Cheers,

ave Salt
SimpList(tm) - try it; you'll get it!
http://www.mackinney.com/products/program-development/simplist.html


> Date: Tue, 7 Feb 2012 12:11:14 -0500
From: robz...@AOL.COM
Subject: Re: LM facilities
To: TSO-...@VM.MARIST.EDU

While we can debate multiple ways of accomplishing the same thing, I tend to

ike to get a benchmark once all the ideas are on the table... Like many of

ou, I have used all the techniques at various times, but never benchmarked them

gainst each other for a common purpose. Yes, I had some time on my hands

actually needed a break from what I was doing to clear my head). I added an

MMFIND/LMGET approach called PDSIO5.

All these tests are running in batch (same job as seen in the prior email).

and here are the new results with a larger dataset with larger members (50%

ver 2000 lines):

PDSIO1 - 1436 members in MY.EXEC.NEW in 0.132425 seconds
PDSIO2 - 1436 members in MY.EXEC.NEW in 21.419272 seconds
PDSIO3 - 1436 members in MY.EXEC.NEW in 53.657526 seconds
PDSIO4 - 1436 members in MY.EXEC.NEW in 30.104675 seconds
PDSIO5 - 1436 members in MY.EXEC.NEW in 31.281468 seconds

Here are the JOBLOG stats...

STEPNAME PROCSTEP PROGRAM RC EXCP CPU SRB CLOCK

ERV
PDSIO1 PDSIO IKJEFT01 00 424 .09 .00 .54

799
PDSIO2 PDSIO IKJEFT01 00 11900 6.77 .07 21.82

66K
PDSIO3 PDSIO IKJEFT01 00 40580 8.35 .23 54.11

73K
PDSIO4 PDSIO IKJEFT01 00 19935 1.10 .11 30.52

3507
PDSIO5 PDSIO IKJEFT01 00 6087 15.35 .04 31.64

15K

Review of what they do...

PDSIO1 - Basic LMMLIST and print member - no lines - for comparison
PDSIO2 - LMMLIST and ALLOC/EXECIO/FREE print number of lines - WINNER
PDSIO3 - LMMLIST/VIEW/.ZLAST Edit Macro print number of lines - 4TH Place
PDSIO4 - LMMSTATS DELETE/LMMSTATS/LMMLIST print number of lines - 2ND Place

by a nose)
PDSIO5 - LMMLIST and LMMFIND/LMGET print number of lines - 3RD place

Analysis:

Surprisingly PDSIO2 LMMLIST/ALLOC/EXECIO/FREE was the fastest by ~30%. This

proach seemd to have the best balance between CPU and I/O.
PDSIO5 LMMLIST/LMMFIND/LMGET was looking the worst in the smaller dataset

rial, but got better with volume and became the preferred second choice over

he LMMSTATS approach since that approach was invasive to the PDS directory and

ossibly impacted other users of the member stats (maybe not for the OP's

roject, but overall). However, the LMMFIND/LMGET approach consumed almost 3

imes the CPU as the EXECIO approach, 15 times the LMMSTATS approach and almost

times the Edit Macro approach. On the flip side, this technique was the

owest EXCP approach
PDSIO3 was the Edit Macro approach and seemed to be the laggard, but was still

espectable. If more processing was done other than extract the last line and

rint the member and line count, I suspect we might have seen an economies of

cale... Interestingly, the Edit Macro approach was the highest EXCP user. It

ould be interesting to know what is different about his I/O technique that

akes him such an I/O pig...


The PDSIO5 code:

000001 /* rexx - PDSIO5 - LMMLIST/LMMFIND/LMGET */
> 000002 x = time('r')
> 000003 arg dsn pat
> 000004 if pat = '' then pat = '*'
> 000005 count = 0
> 000006 address ISPEXEC
> 000007 "LMINIT DATAID(PDS) DATASET('"dsn"') ENQ(SHR)"
> 000008 "LMOPEN DATAID("pds")"
> 000009 do forever
> 000010 "LMMLIST DATAID("pds") OPTION(LIST) STATS(NO)",
> 000011 "MEMBER(ZLMEMBER) PATTERN("pat")"
> 000012 if RC <> 0 then leave
> 000013 count = count + 1
> 000014 "LMMFIND DATAID("pds") MEMBER("strip(zlmember)") LRECL(MRECL)"
> 000015 if RC = 0 then
> 000016 linecnt = 0
> 000017 do forever
> 000018 "LMGET DATAID("pds") MODE(INVAR) DATALOC(RECVAR)

ATALEN(RECLEN)",
000019 "MAXLEN("mrecl")"
> 000020 if RC = 8 then
> 000021 leave
> 000022 else
> 000023 linecnt = linecnt + 1
> 000024 end
> 000025 say left(zlmember,8) linecnt 'lines'
> 000026 end
> 000027 "LMMLIST DATAID("pds") OPTION(FREE)"
> 000028 "LMCLOSE DATAID("pds")"
> 000029 "LMFREE DATAID("pds")"
> 000030 say count 'members in' dsn 'in' time('e') 'seconds'
>

Now, it's back to the salt mines...

And YES, my email client sucks. AOL Web interface... That being said, I have

easons I stay with AOL (occasional contacts with consulting clients that

[Sebastian Welton]

Would something this help, works fine for me:

xxx = Outtrap('al.')
'FREE FI(INDSN2)'
'FREE FI(IN OUT TOOLMSG DFSMSG)'
'ALLOC FI(TOOLMSG) DA(*)'
'ALLOC FI(DFSMSG) DA(*)'
"ALLOC FI(IN) DA('INPUT.DSN') SHR"
"ALLOC FI(OUT) DA('SORT1.DSN') SHR"
"ALLOC FI(DISCARD) DA('SORT2.DSN') SHR"
'ALLOC FI(TOOLIN) NEW TRACKS SPACE(1 1) RECFM(F B) LRECL(80)'
xxx = Outtrap('OFF')
Push ' SELECT FROM(IN) TO(OUT) ON(1,146,CH) DISCARD(DISCARD) ALLDUPS'
'EXECIO 1 DISKW TOOLIN (FINIS'
'CALL *(ICETOOL)'

Thanks,

Sebastian

[Sebastian Welton]

Possibly not what you want but what about using SUPERC with a listing type of OVSUM in batch by comparing the dataset with itself, write the output to a temporary dataset and then a small REXX just to read this information as this shows how many records in a member of a dataset:

MEMBERS-COMPARED NEW-PROC OLD-PROC MATCHED

$$INDEX 107 107 107

[Mickey]

I wasn't thinking of EXECIO, I was thinking of an edit macro, using ISPF
edit to open the files in VIEW mode. I have done this over VERY large PDS
with very large members, and runningi nbatch, they fly.

--------------------------------------------------
From: "Bob Bridges" <rhb...@ATTGLOBAL.NET>

Sent: Tuesday, February 07, 2012 11:13 AM

To: <TSO-...@VM.MARIST.EDU>
Subject: Re: [TSO-REXX] LM facilities

[Thomas Berg]

It *may* be more effective (less cpu) to change PDSI02 from:

> 000014 address TSO "ALLOC F(MEM) DA('"pdsmem"') SHR REUSE"
> 000015 address TSO "EXECIO * DISKR MEM (STEM MEM. FINIS"
> 000016 address TSO "FREE F(MEM)"
> 000017 say left(zlmember,8) mem.0 'lines'

To:

> 000014 address TSO "ALLOC F(MEM) DA('"pdsmem"') SHR REUSE"

> 000015 address TSO "EXECIO * DISKR MEM (FINIS"
> 000017 say left(zlmember,8) Queued() 'lines'

Here I:
1: put the records on the stack --> No creation of stem vars
2: Leaved the FREE out of the loop --> No need as You use "REUSE" in the ALLOC.



Regards,
Thomas Berg
_________________________________________
Thomas Berg Specialist A M SWEDBANK



> -----Ursprungligt meddelande-----
> Från: TSO REXX Discussion List [mailto:TSO-...@VM.MARIST.EDU] För Robert
> Zenuk
> Skickat: den 7 februari 2012 18:11
> Till: TSO-...@VM.MARIST.EDU
> Ämne: Re: [TSO-REXX] LM facilities

[Paul Gilmartin]

On Feb 13, 2012, at 03:20, Thomas Berg wrote:
>
>> 000014 address TSO "ALLOC F(MEM) DA('"pdsmem"') SHR REUSE"
>> 000015 address TSO "EXECIO * DISKR MEM (FINIS"
>> 000017 say left(zlmember,8) Queued() 'lines'
>
> Here I:
> 1: put the records on the stack --> No creation of stem vars

o But then I'd recommend a DROPBUF within the loop to avoid
stack overflow for large aggregate size, or at best unwanted
page I/O.

o Is stack less overhead than stem?

o Regardless, since the OP wanted not a list of size of each
member, but only a list of those members exceding a limit
size (4000, IIRC), I'd use:

address MVS 'EXECIO 4000 DISKR MEM ( FINIS SKIP )'

... and check for RC(2), entirely avoiding storing the data.

-- gil