Performance of EDIT Macros

[John Hooper]

We are not a sophisticated ISPF shop. Most of the EDIT macros we have
either affect small amounts of data or are infrequently used. One of the
infrequently used macros we use takes a list of datasets and creates JCL to
compress each dataset. I had a list of 625 datasets and executed the EDIT
macro to alter the format of each line. The process took 14 minutes!
Either EDIT macros are VERY inefficient or I am doing something in it I
shouldn't. I am sure there are better ways to do what I am doing, but a
relatively small EDIT macro shouldn't take this long. We have several
similar macros that perform similarly poorly. These were developed years
ago and I remember them running relatively quickly then. We are z/OS 1.4
today. I am copying the EDIT macro for all to see - sort of like showing
my dirty laundry. Can anyone tell me what I am doing wrong. Does REXX run
better as an EDIT macro?

ISREDIT MACRO
/* */
/* THIS EDIT MACRO WILL CREATE A COMPRESS STEP FOR EACH */
/* DATASET IN LISTED IN COLUMN 1 OF THE INPUT DATA. */
/* */
/* NOTE THAT NUMBERED DATASETS WILL BE RENUMBERED */
/* */
ISREDIT (MODE,TYPE) = NUMBER
IF &MODE = ON THEN ISREDIT UNNUM
ISREDIT DOWN MAX
ISREDIT (FLINE,LLINE) = DISPLAY_LINES
ISREDIT UP MAX
SET LCTR = 1
SET LOOPCTR = 0
DO WHILE (&LOOPCTR < &LLINE)
SET CNTN1 = 1000 + &LOOPCTR + 1
SET CNTC1 = &SUBSTR(2:4,&STR(&CNTN1))
ISREDIT LOCATE &LCTR
ISREDIT (LINEDATA) = LINE &LCTR
ISREDIT CURSOR = &LCTR 1
ISREDIT SHIFT ) &LCTR 1
ISREDIT CHANGE ' ' "'"
ISREDIT CHANGE ' ' "'"
ISREDIT SHIFT ) &LCTR 1
ISREDIT CURSOR = &LCTR 1
ISREDIT CHANGE ' ' '//STEP&STR(&CNTC1) EXEC COMP,V=SYSRTS,D=' 1
SET LCTR = &LCTR + 1
SET LOOPCTR = &LOOPCTR + 1
END
ISREDIT LOCATE 1
IF &MODE = ON THEN ISREDIT RENUM
EXIT CODE(0)

Thanks,

John Hooper

[Ken MacKenzie]

John,

I don't imagine that REXX would run any quicker but i just might be a bit
easier to follow.

Looking at your CLIST, I can see a few possible inefficiencies.

1. You start by doing "DOWN MAX" followed closely by "UP MAX" - I don't
think that these achieve anything but they're (probably) not a huge
overhead.
2. You do a "LOCATE &LCTR" - again, I think that's unnecessary.
3. You do "(LINEDATA) = LINE &LCTR" - I think this is probably where most
of your processing time is going. You are getting the contents of a line
into an ISPF variable but you don't do anything with it.
4. Not really sure what you're trying to achieve with the "SHIFT, CHANGE,
CHANGE, SHIFT" sequence but that looks like another source of inefficiency

Regards,

Ken

John Hooper <jho...@FOODLION.COM>
To: ISP...@LISTSERV.ND.EDU
Sent by: ISPF discussion list cc:
<ISP...@LISTSERV.ND.EDU> Subject: Performance of EDIT Macros

Thursday January 22, 2004 03:21 PM
Please respond to ISPF discussion
list

[Ray, Ian]

(... I think I have the syntax wrong in the second ISREDIT, but you get the
idea)

-----Original Message-----
From: Ray, Ian
Sent: 22 January 2004 15:48
To: 'ISPF discussion list'
Subject: RE: Performance of EDIT Macros

John,

14 minutes sounds very steep indeed. Your major overhead is the multiple
commands (shift, change) executed against each line. You would do better to
re-construct the line in your code and simply replace it, i.e:

ISREDIT (LINEDATA) = LINE &LCTR

..
.. < Clist statements to reformat &LINEDATA >
..
ISREDIT LINE &LCTR = (LINEDATA)

HTH,
Ian

-----Original Message-----
From: John Hooper [mailto:jho...@FOODLION.COM]
Sent: 22 January 2004 15:22
To: ISP...@LISTSERV.ND.EDU
Subject: Performance of EDIT Macros

Thanks,

John Hooper

---------------------------------------------------------------------------------------------------------------
This e-mail is intended only for the above addressee. It may contain
privileged information. If you are not the addressee you must not copy,
distribute, disclose or use any of the information in it. If you have
received it in error please delete it and immediately notify the sender.

evolvebank.com is a division of Lloyds TSB Bank plc.
Lloyds TSB Bank plc, 25 Gresham Street, London, EC2V 7HN. Registered in
England, number 2065. Telephone No: 020 7626 1500
Lloyds TSB Scotland plc, Henry Duncan House, 120 George Street,
Edinburgh EH2 4LH. Registered in Scotland, number 95237. Telephone
No: 0131 225 4555

Lloyds TSB Bank plc and Lloyds TSB Scotland plc are authorised and
regulated by the Financial Services Authority and represent only the
Scottish Widows and Lloyds TSB Marketing Group for life assurance,
pensions and investment business.

Signatories to the Banking Codes.
---------------------------------------------------------------------------------------------------------------

[Rob Zenuk]

If I interpreted the requirements correctly, here is a REXX version that ran
pretty quick against 650 datasets for me (subsecond)...

/* REXX */
address ISREDIT
"MACRO"
"(LAST) = LINENUM .ZLAST"
do i=1 To last
"(LINE) = LINE" i
"SHIFT )" i "1"
"CHANGE ' ' '//STEP"i" EXEC COMP,V=SYSRTS,D=' 1"
end
"UP MAX"
exit(1)

Hope This Helps,

Rob

[Binyamin Dissen]

On Thu, 22 Jan 2004 10:21:50 -0500 John Hooper <jho...@FOODLION.COM> wrote:

:>We are not a sophisticated ISPF shop. Most of the EDIT macros we have

:>either affect small amounts of data or are infrequently used. One of the
:>infrequently used macros we use takes a list of datasets and creates JCL to
:>compress each dataset. I had a list of 625 datasets and executed the EDIT
:>macro to alter the format of each line. The process took 14 minutes!
:>Either EDIT macros are VERY inefficient or I am doing something in it I
:>shouldn't. I am sure there are better ways to do what I am doing, but a
:>relatively small EDIT macro shouldn't take this long. We have several
:>similar macros that perform similarly poorly. These were developed years
:>ago and I remember them running relatively quickly then. We are z/OS 1.4
:>today. I am copying the EDIT macro for all to see - sort of like showing
:>my dirty laundry. Can anyone tell me what I am doing wrong. Does REXX run
:>better as an EDIT macro?

Possibilities:

ISPF not in LPA
Penalty performance group

Also, you can reduce the number of commands.

As the dsn is in C1 you can simply build the statement in one line using the
LINEDATA variable rather than the interesting series of shifts and changes.

--
Binyamin Dissen <bdi...@dissensoftware.com>
http://www.dissensoftware.com

Director, Dissen Software, Bar & Grill - Israel

[Ray, Ian]

John,

14 minutes sounds very steep indeed. Your major overhead is the multiple
commands (shift, change) executed against each line. You would do better to
re-construct the line in your code and simply replace it, i.e:

ISREDIT (LINEDATA) = LINE &LCTR
..
.. < Clist statements to reformat &LINEDATA >
..
ISREDIT LINE &LCTR = (LINEDATA)

HTH,
Ian

-----Original Message-----
From: John Hooper [mailto:jho...@FOODLION.COM]
Sent: 22 January 2004 15:22
To: ISP...@LISTSERV.ND.EDU
Subject: Performance of EDIT Macros

We are not a sophisticated ISPF shop. Most of the EDIT macros we have
either affect small amounts of data or are infrequently used. One of the
infrequently used macros we use takes a list of datasets and creates JCL to
compress each dataset. I had a list of 625 datasets and executed the EDIT
macro to alter the format of each line. The process took 14 minutes!
Either EDIT macros are VERY inefficient or I am doing something in it I
shouldn't. I am sure there are better ways to do what I am doing, but a
relatively small EDIT macro shouldn't take this long. We have several
similar macros that perform similarly poorly. These were developed years
ago and I remember them running relatively quickly then. We are z/OS 1.4
today. I am copying the EDIT macro for all to see - sort of like showing
my dirty laundry. Can anyone tell me what I am doing wrong. Does REXX run
better as an EDIT macro?

ISREDIT MACRO

[Schwarzbauer, Joe]

I'm not CLIST-literate, but here's a crack at:

.. < Clist statements to reformat &LINEDATA >

SET LINEDATA = &LINEDATA /* TRY TO STRIP TRAILING BLANKS
SET LINEDATA = &STR(//STEP&STR(&CNTC1) EXEC COMP,V=SYSRTS,D='&LINEDATA')

-----Original Message-----
From: Ray, Ian
Sent: 22 January 2004 15:48
To: 'ISPF discussion list'
Subject: RE: Performance of EDIT Macros

John,

14 minutes sounds very steep indeed. Your major overhead is the multiple
commands (shift, change) executed against each line. You would do better to
re-construct the line in your code and simply replace it, i.e:

ISREDIT (LINEDATA) = LINE &LCTR
..
.. < Clist statements to reformat &LINEDATA >
..
ISREDIT LINE &LCTR = (LINEDATA)

HTH,
Ian

-----------------------------------------
CONFIDENTIALITY NOTICE: The information contained in this e-mail and attached document(s) may contain confidential information that is intended only for the addressee(s). If you are not the intended recipient, you are hereby advised that any disclosure, copying, distribution or the taking of any action in reliance upon the information is prohibited. If you have received this e-mail in error, please immediately notify the sender and delete it from your system.

[Rob Zenuk]

In a message dated 1/22/2004 9:32:09 AM US Mountain Standard Time,

bdi...@DISSENSOFTWARE.COM writes:
As the dsn is in C1 you can simply build the statement in one line using the
LINEDATA variable rather than the interesting series of shifts and changes.

Good point, here is a version to do that instead (no shifts, no changes, much
faster)

/* REXX */
address ISREDIT
"MACRO"
"(LAST) = LINENUM .ZLAST"
do i=1 To last
"(LINE) = LINE" i

dsn = word(strip(line),1)
newline = '//S'right(i,7,0) 'EXEC COMP,V=SYSRTS,D='dsn
"LINE" i "=" "(NEWLINE)"
end
"UP MAX"
exit(1)

This version affords some more capabilities. The DSN does not have to be in
column 1. This will pluck out the first word of the record and assume it is
the DSN. This means you can use something like 3.4 SAVE and edit the LIST
dataset as is. Well, I still like to remove all extraneous lines (ex all;f all
'rzenuk.';del all x) which could be incorporated in the edit macro too.

Rob

[Dave Salt]

Hi John,

I don't have time to look at the macro in detail right now, but my
suggestion right off the bat would be to avoid uisng a loop and just use a
simple sequence of CHANGE ALL instructions. This avoids scrolling around,
capturing line contents, determining number of lines in the file, processing
each line one at a time, and so on. In other words, efficiency of the macro
will improve DRAMATICALLY.

As an example, if all of the data sets are lined up in column 1 and you just
want to add a close quote to the end of every data set, the instruction
would be:

ISREDIT CHANGE P"^ " P"='" ALL

This picture-string instruction means "change all occurrences of a non-blank
character followed by a blank character to the same non-blank character
followed by a single quote". In other words, it changes the trailing
character of every data set name in the file to the same trailing character
followed by a quote.

You can use a similar technique to add what you want in front of each data
set name. Who knows, maybe you can get the entire macro down to just a
couple of CHANGE ALL instructions. In any case, by avoiding using a loop the
macro will execute in a matter of seconds or maybe even a fraction of a
second. If I had more time to spend on it I'd put the instructions together
for you, but meanwhile I hope this helps!

Dave Salt
Soft-Center Solutions Inc.
http://www.soft-center.com
1-877-SoftCen
Bringing you SimpList(tm) - The easiest, most powerful way to surf a
mainframe!

> ISREDIT (LINEDATA) = LINE &LCTR

> ISREDIT CURSOR = &LCTR 1
> ISREDIT SHIFT ) &LCTR 1
> ISREDIT CHANGE ' ' "'"
> ISREDIT CHANGE ' ' "'"
> ISREDIT SHIFT ) &LCTR 1
> ISREDIT CURSOR = &LCTR 1
> ISREDIT CHANGE ' ' '//STEP&STR(&CNTC1) EXEC COMP,V=SYSRTS,D=' 1
> SET LCTR = &LCTR + 1
> SET LOOPCTR = &LOOPCTR + 1
>END
>ISREDIT LOCATE 1
>IF &MODE = ON THEN ISREDIT RENUM
>EXIT CODE(0)
>
>Thanks,
>
>John Hooper

_________________________________________________________________
Protect your PC - get McAfee.com VirusScan Online
http://clinic.mcafee.com/clinic/ibuy/campaign.asp?cid=3963

[John Hooper]

Thanks to all that responded to my question. In testing all of the
suggested solutions I discovered what is an even more "interesting"
problem. REXX edit macros run "like a scalded dog". CLIST edit macros run
like a "dead dog"!!! Yes, my CLIST was inefficient and after using some of
the solutions, I got it to run in a little over 3 minutes. A REXX solution
run in about 1 second!

I then stripped the CLIST down to a very basic function of reading each
line and appending 'NEW' in front of the character string at the beginning
of each line. I then converted it to REXX. One ran 3 minutes the other
ran 1 second. See the examples that follow. From all of the comments,
this is totally unexpected. Is this an ISPF problem or a TSO setup problem?

CLIST

ISREDIT MACRO
ISREDIT (LF) = LINENUM .ZFIRST
ISREDIT (LL) = LINENUM .ZLAST

ISREDIT (MODE,TYPE) = NUMBER
IF &MODE = ON THEN ISREDIT UNNUM

DO WHILE (&LF <= &LL)
ISREDIT (DATA) = LINE &LF
SET DATA = &STR(NEW&DATA)
ISREDIT LINE &LF = &DATA
SET &LF = &LF + 1

END
ISREDIT LOCATE 1
IF &MODE = ON THEN ISREDIT RENUM
EXIT CODE(0)

REXX

/* REXX */
ADDRESS ISREDIT
"MACRO"
"(LF) = LINENUM .ZFIRST"
"(LL) = LINENUM .ZLAST"
"(MODE,TYPE) = NUMBER"
IF MODE = 'ON' THEN "UNNUM"
DO I=LF TO LL
"(DATA) = LINE" LF
DATA='NEW'||DATA
"LINE" LF "=" DATA
LF = LF+1
END
"LOCATE 1"
IF MODE = 'ON' THEN "RENUM"
EXIT(1)

[John Kalinich]

> From: John Hooper [mailto:jho...@FOODLION.COM]

> Is this an ISPF problem or a TSO setup problem?

Binyamin Dissen has the right answer. Ask your MVS performance
management group.

Possibilities:

Penalty performance group

[Binyamin Dissen]

On Thu, 22 Jan 2004 13:32:20 -0500 John Hooper <jho...@FOODLION.COM> wrote:

:>Thanks to all that responded to my question. In testing all of the

:>suggested solutions I discovered what is an even more "interesting"
:>problem. REXX edit macros run "like a scalded dog". CLIST edit macros run
:>like a "dead dog"!!! Yes, my CLIST was inefficient and after using some of
:>the solutions, I got it to run in a little over 3 minutes. A REXX solution
:>run in about 1 second!

:>I then stripped the CLIST down to a very basic function of reading each
:>line and appending 'NEW' in front of the character string at the beginning
:>of each line. I then converted it to REXX. One ran 3 minutes the other
:>ran 1 second. See the examples that follow. From all of the comments,
:>this is totally unexpected. Is this an ISPF problem or a TSO setup problem?

Looks like the ISREDIT load module is not in a good place. Is it in LPA?

Also, overhead from attaching it. See what happens if it is added to
IKJTSO/PLATCMD.

[Hooper, John]

Thanks for the response.

We DO have ISPF in LPA including ISREDIT. I put ISREDIT in IKJTSO PLATCMD
and PLATPGM as well. Same results. Any other ideas? Could you
cut-and-paste my sample for CLIST and try it on a member that has a couple
of hundred dataset type entries starting in column 1? It would be great to
know if this is only happening here.

John

This electronic message may contain confidential or privileged information
and is intended for the individual or entity named above. If you are
not the intended recipient, be aware that any disclosure, copying,
distribution or use of the contents of this information is prohibited.
If you have received this electronic transmission in error, please notify
the sender immediately by using the e-mail address or by telephone
(704-633-8250).

[Hooper, John]

I found the problem. It is related to your solution. Our LOGON proc has 2
STEPLIB libraries and 10 ISPLLIB libraries. If I logon without the STEPLIB
and free the ISPLLIB libraries, the CLIST runs in about 2 seconds. I have
to free both of them. Obviously CLIST processing searches those
concatinations and REXX doesn't. Good to know. Another reason to go to
dynamic ISPF. This issue is certainly not intuitive.

I will post the list with this answer. Thanks for your help.

[Mark Zelden]

On Thu, 22 Jan 2004 17:19:03 -0500, Hooper, John <JHo...@FOODLION.COM>
wrote:

>I found the problem. It is related to your solution. Our LOGON proc has 2
>STEPLIB libraries and 10 ISPLLIB libraries. If I logon without the STEPLIB
>and free the ISPLLIB libraries, the CLIST runs in about 2 seconds. I have
>to free both of them. Obviously CLIST processing searches those
>concatinations and REXX doesn't. Good to know. Another reason to go to
>dynamic ISPF. This issue is certainly not intuitive.
>

STEPLIBs in a logon proc are a big no-no. Every time you hit enter
the STEPLIBs are searched. They are okay for testing, but you should
try and get rid of them by placing the libraries in the LNKLST if
they can't be used in ISPLLIB. It's not just hurting this macro, its
hurting all of your ISPF environment since most of the modules come
from LPA and LNKLST. Since ISPLLIB is a tasklib it is also searched
prior to STEPLIB. It's best to put them in the LNKLST if you can
also. If you can't elminiate them easily, you can still
add them to LLA (via CSVLLAxx parmlib member) with FREEZE and that
will help since the directory will be in memory.

--
Mark Zelden
Sr. Software and Systems Architect
mailto: mark....@zurichna.com
Systems Programming expert at http://Search390.com/ateExperts/
Mark's MVS Utilities: http://home.flash.net/~mzelden/mvsutil.html

[Leonard Woren]

On Thu, Jan 22, 2004 at 05:19:03PM -0500, Hooper, John (JHo...@FOODLION.COM) wrote:
> I found the problem. It is related to your solution. Our LOGON proc has 2
> STEPLIB libraries and 10 ISPLLIB libraries.

Gag. While I typically have a STEPLIB on my personal logon proc
because I need it for testing my TSO commands before they move to
linklist, ISPLLIB is supposed to application-specific. As someone
else subsequently posted, ISPLLIB in a logon proc should be a no-no.
Applications that need ISPLLIB should LIBDEF it for the duration of
that application. Otherwise it's unmanageable. And the IBM-supplied
logon procs with ISPLLIB for IBM product libraries are ridiculous.

When ISPLLIB came out, it was clearly stated that it was for
application development use only, not for production. LIBDEF
addresses that, but permanent ISPLLIBs I think are still something
on the "don't do this" list.

> If I logon without the STEPLIB
> and free the ISPLLIB libraries, the CLIST runs in about 2 seconds. I have
> to free both of them. Obviously CLIST processing searches those
> concatinations and REXX doesn't. Good to know. Another reason to go to
> dynamic ISPF. This issue is certainly not intuitive.

Interesting. That must somehow have to do with the fact that a CLIST
is basically a series of commands and each command has to be located,
while a REXX exec is an actual program. There are probably major
differences in how the ISREDIT interfaces work between CLIST and REXX.
Note in particular that in REXX, you can't say "Address TSO ISREDIT",
so it's definitely not going through the standard TSO command search
path.

One more reason to not use CLIST.

/Leonard

[Binyamin Dissen]

On Thu, 22 Jan 2004 17:19:03 -0500 "Hooper, John" <JHo...@FOODLION.COM>
wrote:

:>I found the problem. It is related to your solution. Our LOGON proc has 2

:>STEPLIB libraries and 10 ISPLLIB libraries. If I logon without the STEPLIB
:>and free the ISPLLIB libraries, the CLIST runs in about 2 seconds. I have
:>to free both of them. Obviously CLIST processing searches those
:>concatinations and REXX doesn't. Good to know. Another reason to go to
:>dynamic ISPF. This issue is certainly not intuitive.

More likely that REXX keep the address of ISREDIT around.

May be different if instead of ADDRESS ISREDIT you did ADDRESS ISPEXEC and
prefaced each command with ISREDIT.

Also, would be interesting to see what happened if you wrote a macro that did
an ISREDIT to your original macro.

:>I will post the list with this answer. Thanks for your help.

:>-----Original Message-----

[Ricc Harding]

John;
I ran your clist unchanged against 1000 datasets and it ran in less than 2
seconds.
I would bet money that most of the time you spend running the clist you are
swapped out.
There are improvements that can be made but if you are not running (swapped
out) then it doesn't matter how fast your clist or exec is.
You can see if you are dropping into 3rd period performance and swapping out
on RMFMON.
You may need to adjust your WLM or performance group service units.
Ricc

----- Original Message -----
From: "Hooper, John" <JHo...@FOODLION.COM>
Newsgroups: bit.listserv.ispf-l
Sent: Thursday, January 22, 2004 5:30 PM
Subject: Re: Performance of EDIT Macros

> I found the problem. It is related to your solution. Our LOGON proc has
2
> STEPLIB libraries and 10 ISPLLIB libraries. If I logon without the
STEPLIB
> and free the ISPLLIB libraries, the CLIST runs in about 2 seconds. I have
> to free both of them. Obviously CLIST processing searches those
> concatinations and REXX doesn't. Good to know. Another reason to go to
> dynamic ISPF. This issue is certainly not intuitive.
>

> I will post the list with this answer. Thanks for your help.
>

> John

***********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. Any unauthorized review, use, disclosure or distribution
is prohibited. If you are not the intended recipient, please contact
the sender by reply e-mail and destroy all copies of the original
message.
***********************************************************************

[Mark Zelden]

On Thu, 22 Jan 2004 17:35:20 -0800, Leonard Woren <ispf...@LDWOREN.NET>
wrote:

>As someone
>else subsequently posted, ISPLLIB in a logon proc should be a no-no.
>Applications that need ISPLLIB should LIBDEF it for the duration of
>that application.

Actually, I said STEPLIBs were a no-no. But the same usually
applies to ISPLLIB too. (BTW, did ya ever notice how IBM used to
ship CPAC and ServerPac with SISPLOAD in the LNKLST and in
the ISPLLIB concatenation... they did fix this some time in the
recent past.)

The only problem is that LIBDEF to ISPLLIB will not work for
LINK,XCTL,LOAD, and ATTACH. So I think for "in-house" applications,
ISPLLIB is the only answer if you don't want the application library
in the LNKLST (and I don't). However, "trusted" ISPLLIBs from ISVs
that are protected by your security product from update to all but
system progammers, can easily be put in the LNKLST without concern.

That being said, the list of "in-house" applications that I've come
across in all my years that would not work with LIBDEF for ISPLLIB
can be counted on one hand.

Mark

[Gilbert Saint-Flour]

On Thu, 22 Jan 2004 13:54:29 -0800, Hooper, John wrote:

> :>Thanks to all that responded to my question. In testing all of the
> :>suggested solutions I discovered what is an even more "interesting"
> :>problem. REXX edit macros run "like a scalded dog". CLIST edit
> :>macros run like a "dead dog"!!!

That's probably because you coded an ADDRESS ISREXEC at the top of the
REXX exec. Could you change the exec to remove ADDRESS ISREXEC and try it
again? I would expect it to run slower, perhaps as slow as the equivalent
CLIST.

What version of ISPF are you running? I can't seem to recreate your
symptoms, and I'm on a P/390 ! Also, what's the number of EXCPs issued by
the CLIST macro? You can see that in the SDSF DA display.

> Looks like the ISREDIT load module is not in a good place. Is it in LPA?

Where ISREDIT is doesn't matter much. What's important is how many
directory tracks have to be scanned before ISPF finds it.

> Also, overhead from attaching it. See what happens if it is added to
> IKJTSO/PLATCMD.

AFAIK, PLATCMD/PLATPGM only apply to programs and commands invoked by the
TMP itself. In a CLIST EDIT macro, ISPF is the one invoking ISREDIT I
believe.

[Hooper, John]

I found the problem. Our LOGON proc has 2 STEPLIB libraries and 10 ISPLLIB

libraries. If I logon without the STEPLIB and free the ISPLLIB libraries,
the CLIST runs in about 2 seconds. I have to free both of them. Obviously
CLIST processing searches those concatinations and REXX doesn't. Good to
know. Another reason to go to dynamic ISPF. This issue is certainly not
intuitive.

FYI: I had an ADDRESS ISREDIT in the REXX. I removed it and found that the
REXX still ran in a couple of seconds??? Seems like REXX "remembers" the
address of ISREDIT and does not try to "find" it on each call.

I tested the I/O counts for both REXX and CLIST. The REXX issued 142 I/O s.
The CLIST issued 30,211! Big difference. Again, it is obviously constantly
going through the directory blocks on STEPLIB and ISPLLIB.

We are running z/OS 1.4 with ISPF 5.2

Thanks to all that worked on this problem. I "knew" that STEPLIBs were a
"no,no" but yielded to other pressures. I now know why they are a "no,no".
On "simple" processes, they are not too bad. In CLISTS they obviously are.
I did NOT realize the issue with an ISPLLIB concatination. It makes sense
but it not as obvious as a STEPLIB. We are certainly going to address this
issue the way it should be using the recommended DYNAMIC ISPF which exploits
LIBDEFs. Again thanks to all.

John

-----Original Message-----
From: Gilbert Saint-Flour [mailto:usene...@YAHOO.COM]
Sent: Friday, January 23, 2004 9:48 AM
To: ISP...@LISTSERV.ND.EDU
Subject: Re: Performance of EDIT Macros

[Mark Zelden]

On Fri, 23 Jan 2004 10:56:03 -0500, Hooper, John <JHo...@FOODLION.COM>
wrote:

>

>Thanks to all that worked on this problem. I "knew" that STEPLIBs were a
>"no,no" but yielded to other pressures. I now know why they are a "no,no".
>On "simple" processes, they are not too bad. In CLISTS they obviously are.

For CLISTs that are not calling programs it is not really an issue except
for the initial time it takes to find the CLIST. You can eliminate that
issue by always invoking your CLISTs with a percent sign (as in %CLIST)
whether it be from other CLISTs or interactivly in TSO/ISPF. The first
library searched will then be SYSPROC instead of the normal program
fetch search order.

A further benefit will be realized by defining the IKJEXEC class in VLF
(COFVLFxx) with your CLIST libraries. The compiled object is what is
stored in VLF. Caution: There are some caveats with doing this if the
library is shared with another system outside a sysplex and it is
updated (so CLIST libraries from ISVs and IBM are great candidates).
Search the ISPF-L archives or IBM-MAIN archives if you want to
consider doing it. I'm sure this has been discussed before.

[BUDBILL, GENE]

You're on the right track with how to solve this problem, but out of
curiosity, what happens if you invoke your CLIST using the %clistname
syntax. Prefixing your clistname with the '%' character tells the
interpreter that you are explicitly executing a CLIST and to restrict
searches to the SYSPROC concatenation and bypasses any I/O to the
steplibs looking for same named command processors.

Gene

-----Original Message-----
From: ISPF discussion list [mailto:ISP...@LISTSERV.ND.EDU] On Behalf Of
Hooper, John
Sent: Friday, January 23, 2004 7:56 AM
To: ISP...@LISTSERV.ND.EDU
Subject: Re: Performance of EDIT Macros

I found the problem. Our LOGON proc has 2 STEPLIB libraries and 10
ISPLLIB libraries. If I logon without the STEPLIB and free the ISPLLIB
libraries, the CLIST runs in about 2 seconds. I have to free both of
them. Obviously CLIST processing searches those concatinations and REXX
doesn't. Good to know. Another reason to go to dynamic ISPF. This
issue is certainly not intuitive.

FYI: I had an ADDRESS ISREDIT in the REXX. I removed it and found that
the REXX still ran in a couple of seconds??? Seems like REXX
"remembers" the address of ISREDIT and does not try to "find" it on each
call.

I tested the I/O counts for both REXX and CLIST. The REXX issued 142
I/O s. The CLIST issued 30,211! Big difference. Again, it is obviously
constantly going through the directory blocks on STEPLIB and ISPLLIB.

We are running z/OS 1.4 with ISPF 5.2

Thanks to all that worked on this problem. I "knew" that STEPLIBs were

a "no,no" but yielded to other pressures. I now know why they are a
"no,no". On "simple" processes, they are not too bad. In CLISTS they

obviously are. I did NOT realize the issue with an ISPLLIB

[Hooper, John]

The '%' character only helps the initial call of the CLIST. Any program
calls from within the CLIST go through STEPLIB and ISPLLIB no matter what.

John

[Mark Zelden]

On Fri, 23 Jan 2004 08:49:30 -0800, BUDBILL, GENE <GEN...@SAFECO.COM> wrote:

> Prefixing your clistname with the '%' character tells the
>interpreter that you are explicitly executing a CLIST and to restrict
>searches to the SYSPROC concatenation and bypasses any I/O to the
>steplibs looking for same named command processors.
>

No, it doesn't restrict the search to SYSPROC, it just changes the
call of that CLIST to search SYSPROC first. After SYSPROC the
normal search order applies: job pack area, TASKLIB(s), STEPLIB,
JOBLIB if present and no STEPLIB (not applicable unless running
in batch), LPA, and LNKLST. Without the percent, SYSPROC is at the end
of the list instead of the front of the list.

Even if you invoke your CLIST with percent, other CLISTs called
from within that CLIST will look at SYSPROC last unless they
are also invoked with a percent.

[Gilbert Saint-Flour]

On Fri, 23 Jan 2004 07:57:51 -0800, Hooper, John wrote:

> I found the problem. Our LOGON proc has 2 STEPLIB libraries and 10
> ISPLLIB libraries. If I logon without the STEPLIB and free the ISPLLIB
> libraries, the CLIST runs in about 2 seconds. I have to free both of
> them. Obviously CLIST processing searches those concatinations and REXX
> doesn't. Good to know. Another reason to go to dynamic ISPF. This
> issue is certainly not intuitive.

There are plenty of good reasons to use Dynamic ISPF, but I'm not sure
CLIST macro performance is one of them.

> FYI: I had an ADDRESS ISREDIT in the REXX. I removed it and found that
> the REXX still ran in a couple of seconds??? Seems like REXX
> "remembers" the address of ISREDIT and does not try to "find" it on each
> call.

I'm surprised. I'm pretty sure it didn't use to be this way, I ran a lot
a tests in this area, but it wat over 10 years ago and I'm not sure about
the details. Do you also have an ADDRESS ISPEXEC by any chance ? IIRC,
ADDRESS ISPEXEC and ADDRESS ISREDIT provide similar performance in a REXX
macro.

> I tested the I/O counts for both REXX and CLIST. The REXX issued 142
> I/O s. The CLIST issued 30,211! Big difference. Again, it is obviously
> constantly going through the directory blocks on STEPLIB and ISPLLIB.

No surprise here.

> We are running z/OS 1.4 with ISPF 5.2

I'm running z/OS 1.2 with ISPF 5.2. Why I can't recreate your symptoms
when I run your original CLIST, I don't know.

> Thanks to all that worked on this problem. I "knew" that STEPLIBs were
> a "no,no" but yielded to other pressures. I now know why they are a
> "no,no". On "simple" processes, they are not too bad. In CLISTS they
> obviously are. I did NOT realize the issue with an ISPLLIB
> concatination. It makes sense but it not as obvious as a STEPLIB. We
> are certainly going to address this issue the way it should be using the
> recommended DYNAMIC ISPF which exploits LIBDEFs. Again thanks to all.

STEPLIB and ISPLLIB are only a performance issue if you issue a lot of
LOAD SVC for modules which are in system libraries. This happens mostly
in EDIT macros written in CLIST, as your example showed.

[Gilbert Saint-Flour]

John,

Link this program and invoke it before you execute the CLIST macro.
Performance should improve.

You can call it any name you want, link it into any load library, and
invoke it as a command or a program any way you like.

--

//GILBERT2 JOB (ACCT#),ISREDIT2,
// CLASS=A,MSGCLASS=X,COND=(0,NE),NOTIFY=&SYSUID //HLASM EXEC
PGM=ASMA90,PARM=(OBJECT,NODECK,NOESD,NORLD,NOXREF) *
* CREATE CDE FOR ISREDIT
*
ISREDIT2 CSECT
ISREDIT2 AMODE ANY
LR R12,R15
USING ISREDIT2,R12
LOAD EP=ISREDIT
LR R2,R0
IDENTIFY EP=ISR_EDIT,ENTRY=(R2)
LTR R15,R15
BNZ ERR1
DELETE LOAD EP=ISR_EDIT
DELETE EP=ISREDIT
IDENTIFY EP=ISREDIT,ENTRY=(R2)
LTR R15,R15
BNZ ERR2
SVC 3
ERR1 TPUT =C'IDENTIFY 1 FAILED',17
SVC 3
ERR2 TPUT =C'IDENTIFY 2 FAILED',17
SVC 3
YREGS
END
//SYSLIB DD DSN=SYS1.MACLIB,DISP=SHR
//SYSUT1 DD UNIT=VIO,SPACE=(CYL,2)
//SYSPRINT DD SYSOUT=*
//SYSLIN DD UNIT=VIO,SPACE=(TRK,1),DISP=(,PASS)
//*
//LKED EXEC PGM=IEWL
//SYSPRINT DD SYSOUT=*
//SYSLIN DD DSN=*.HLASM.SYSLIN,DISP=(OLD,DELETE)
//SYSLMOD DD DSN=GILBERT.LOAD2(ISREDIT2),DISP=SHR

[Frank Clarke]

On 22 Jan 2004 10:44:17 -0800, jho...@FOODLION.COM (John Hooper)
wrote:
<200401221832....@pickering.cc.nd.edu>

>Thanks to all that responded to my question. In testing all of the
>suggested solutions I discovered what is an even more "interesting"
>problem. REXX edit macros run "like a scalded dog". CLIST edit macros run
>like a "dead dog"!!! Yes, my CLIST was inefficient and after using some of
>the solutions, I got it to run in a little over 3 minutes. A REXX solution
>run in about 1 second!
>
>I then stripped the CLIST down to a very basic function of reading each
>line and appending 'NEW' in front of the character string at the beginning
>of each line. I then converted it to REXX. One ran 3 minutes the other
>ran 1 second. See the examples that follow. From all of the comments,
>this is totally unexpected. Is this an ISPF problem or a TSO setup problem?

I believe that CLIST, since it runs only in what REXX programmers
think of as 'address TSO', has no option but to continually dispatch
fresh tasks for each operation. REXX, on the other hand, is able to
deliver a command directly to an existing environment and does not
incur the prolog-epilog costs of each action.

When you code as
address TSO
"ISPEXEC VGET ZORK SHARED"
you're probably no more efficient than a CLIST since what you have
effectively said is "Hey, TSO! Ask ISPEXEC to VGET...."

(change Arabic number to Roman numeral to email)

[Heinz-Bernd Leifeld]

Hi list,

I don't know, what Gilbert is doing in the above example, but it may be
the same that IRXINIT does under ISPF: Prefetch ISPEXEC and ISREDIT. See
sys1.samplib(IRXREXX3). So they become part of the Rexx-environment and
need not to be loaded any more. We came across this when playing with
DSNREXX which showed poor performance until we put it in IRXTSPRM (see
SYS1.SAMPLIB(IRXREXX2)) and it sped up like a rocket.

Regards
Heinz-Bernd

--
Provinzial Rheinland Versicherung AG
Heinz-Bernd Leifeld
Abtlg.: KIP 3 / ETS C00
D-40195 Düsseldorf

[Mickey]

JHo...@FOODLION.COM (Hooper, John) wrote in message news:<3773C7BF491F2E45936...@flexch01.foodlion.ad.delhaize.com>...

> I found the problem. Our LOGON proc has 2 STEPLIB libraries and 10 ISPLLIB
> libraries. If I logon without the STEPLIB and free the ISPLLIB libraries,
> the CLIST runs in about 2 seconds. I have to free both of them. Obviously
> CLIST processing searches those concatinations and REXX doesn't. Good to
> know. Another reason to go to dynamic ISPF. This issue is certainly not
> intuitive.
>

One additional piece of advice I would give is to put your REXX in a
librarby concatenated to SYSEXEC and not SYSPROC.

Mickey

[Jeremy C B Nicoll]

In article <2F4FF3A58E8BD211A28C...@hqexch.serena.com>,

Ricc Harding <RHar...@SERENA.COM> wrote:
> John; I ran your clist unchanged against 1000 datasets and it ran in
> less than 2 seconds. I would bet money that most of the time you
> spend running the clist you are swapped out.

I was having similar thoughts - none of the minor coding changes people
have suggested should make any real difference (in this case, which is
simple manipulation of a list of dsnames). One thing that might help,
in more complicated macros, is making sure that edit recovery is turned
off as you edit the file, to stop ispf from building a step-by-step
recovery process.

Ages ago, when working with a product called TSOMON, I seem to remmber
there being a problem that each individual line of a clist would be
treated as a single TSO transaction. I can't remember if this depended
on some overall system setting or not - but it meant that some clists
ran very slowly because the address space got swapped out multiple
times, rather than (as you'd hope) only when the total amount of
resource used executing a clist got past some threshhold.

--
Jeremy C B Nicoll - my opinions are my own.

[Mark Zelden]

On Tue, 27 Jan 2004 21:55:31 +0000, Jeremy C B Nicoll
<Jer...@OMBA.DEMON.CO.UK> wrote:

>Ages ago, when working with a product called TSOMON, I seem to remmber
>there being a problem that each individual line of a clist would be
>treated as a single TSO transaction. I can't remember if this depended
>on some overall system setting or not - but it meant that some clists
>ran very slowly because the address space got swapped out multiple
>times, rather than (as you'd hope) only when the total amount of
>resource used executing a clist got past some threshhold.
>

CNTCLIST=YES|NO in IEAOPTxx. It is still valid - even under goal
mode. In my experience, you get better performance for
clists since you start over in TSO period 1 at every new command.

There is this warning in the MVS Init and Tuning Guide:
"A possible exposure of specifying CNTCLIST=YES is that long CLISTs
composed of trivial and intermediate commands might monopolize a
domain's MPL slots and cause interactive terminal users to be delayed."

This is not a concern for goal mode and it unless you specified
contraints in your domains for COMPAT mode, it shouldn't have been
an issue.

I have heard of some consultants who went into shops and changed
CNTCLIST from NO (the default) to YES and then gave managers
RMF reports that showed they increased the amount of TSO transactions
getting through the system in a given time period with their
tuning effort.

[Mark Zelden]

On Wed, 28 Jan 2004 16:25:31 -0500, Mark Zelden <mark....@ZURICHNA.COM>
wrote:

>
>CNTCLIST=YES|NO in IEAOPTxx. It is still valid - even under goal
>mode. In my experience, you get better performance for
>clists since you start over in TSO period 1 at every new command.
>

I forgot to mention one of the main reasons I like CNTCLIST=YES:
Quicker LOGON to TSO - especially if you have a large LOGON CLIST with
a lot of ALLOCATE commands. If the CPU is pegged it could
take a long time to logon if you drop into the last period of
TSO while the CLIST runs with CNTCLIST=NO.

[McKown, John]

This specifically says CLIST. Does it work the same way for a REXX exec?

--
John McKown
Senior Systems Programmer
UICI Insurance Center
Applications & Solutions Team
+1.817.255.3225

This message (including any attachments) contains confidential information
intended for a specific individual and purpose, and its' content is
protected by law. If you are not the intended recipient, you should delete
this message and are hereby notified that any disclosure, copying, or
distribution of this transmission, or taking any action based on it, is
strictly prohibited.

> -----Original Message-----
> From: Mark Zelden [mailto:mark....@ZURICHNA.COM]
> Sent: Wednesday, January 28, 2004 3:33 PM
> To: ISP...@LISTSERV.ND.EDU
> Subject: Re: Performance of EDIT Macros
>
>

[Mark Zelden]

On Wed, 28 Jan 2004 15:36:55 -0600, McKown, John
<john....@UICIINSCTR.COM> wrote:

>This specifically says CLIST. Does it work the same way for a REXX exec?
>

No. I just confirmed it with a quick test - a rexx exec with a bunch
of TSO ALLOC commands in it.

Mark