Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

'Inexperienced' RBS tech operative's blunder led to banking meltdown

182 views
Skip to first unread message

Ed Gould

unread,
Jun 27, 2012, 12:44:08 AM6/27/12
to
http://www.theregister.co.uk/2012/06/26/
rbs_natwest_ca_technologies_outsourcing/


----------------------------------------------------------------------
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to list...@listserv.ua.edu with the message: INFO IBM-MAIN

Tom Ambros

unread,
Jun 27, 2012, 8:25:57 AM6/27/12
to
"Complicated legacy mainframe system"? It's CA-7! If they think that's
complicated they probably can't work a Mr. Coffee.

Thomas Ambros
Operating Systems and Connectivity Engineering
518-436-6433
This communication may contain privileged and/or confidential information. It is intended solely for the use of the addressee. If you are not the intended recipient, you are strictly prohibited from disclosing, copying, distributing or using any of this information. If you received this communication in error, please contact the sender immediately and destroy the material in its entirety, whether electronic or hard copy. This communication may contain nonpublic personal information about consumers subject to the restrictions of the Gramm-Leach-Bliley Act. You may not directly or indirectly reuse or redisclose such information for any purpose other than to provide the services for which you are receiving the information.

127 Public Square, Cleveland, OH 44114
If you prefer not to receive future e-mail offers for products or services from Key
send an e-mail to mailto:DNERe...@key.com with 'No Promotional E-mails' in the
SUBJECT line.

McKown, John

unread,
Jun 27, 2012, 8:34:29 AM6/27/12
to
If it's not "point and click", then it is beyond today's average person. I have envisioned a way to make so much money, I could bail out the Eurozone. Make a UI which is modelled on the FRPG "first person shooter" interfaces. "Frag that accounts receivable information!". After all, it is what the majority of kids today are learning. "Accounting for Nintendo DS User" would be a best seller (audio format - rap style, but with optional epub or mobi, for the few who can read)

--
John McKown
Systems Engineer IV
IT

Administrative Services Group

HealthMarkets(r)

9151 Boulevard 26 * N. Richland Hills * TX 76010
(817) 255-3225 phone *
john....@healthmarkets.com * www.HealthMarkets.com

Confidentiality Notice: This e-mail message may contain confidential or proprietary information. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. HealthMarkets(r) is the brand name for products underwritten and issued by the insurance subsidiaries of HealthMarkets, Inc. -The Chesapeake Life Insurance Company(r), Mid-West National Life Insurance Company of TennesseeSM and The MEGA Life and Health Insurance Company.SM

Scott Ford

unread,
Jun 27, 2012, 8:47:21 AM6/27/12
to
John,

Count me in...

Scott ford
www.identityforge.com

John Gilmore

unread,
Jun 27, 2012, 8:55:02 AM6/27/12
to

Knutson, Sam

unread,
Jun 27, 2012, 9:49:28 AM6/27/12
to
To someone who is inexperienced on platform I imagine managing a CA-7 upgrade is like having the world's smartest dog but it only responds to commands in Latin. Miscommunication is likely to be a source of dissatisfaction.

The average person <30 is not stupid because they haven't been editing command decks and JCL for 25 years or lazy but software on our platform tends to be very particular and sometimes unforgiving of mistakes i.e. failure to specify QUEUE=NOFORMAT or some non-obvious phrase as part of START command. Since business is no longer content to bring in newbies and let them spend 5-10 years to reach Journeyman status under the guidance of more senior folks software is going to have to get smarter and more forgiving, documentation is going to have to be written with less assumptions about the expertise of the "systems programmer" and zNextGen (generic) is going to operate "our" mainframes weather we think they are ready or not. Some of those next generation folks are not going to be based in the home office anymore either whether we like that or not.

The Register is fun reading sometimes but they rarely have enough detail to get real insight into what actually happened. So really anything we say here is just idle speculation. Feels like Friday so that's my .02 worth of idle speculation :-)

        Best Regards,

               Sam Knutson, GEICO 
                System z Team Leader 
                mailto:sknu...@geico.com
                (office)  301.986.3574
                (cell) 301.996.1318   
          
"Think big, act bold, start simple, grow fast..."


-----Original Message-----
From: IBM Mainframe Discussion List [mailto:IBM-...@LISTSERV.UA.EDU] On Behalf Of Tom Ambros
Sent: Wednesday, June 27, 2012 8:25 AM
To: IBM-...@LISTSERV.UA.EDU
Subject: Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

"Complicated legacy mainframe system"? It's CA-7! If they think that's complicated they probably can't work a Mr. Coffee.

Thomas Ambros
Operating Systems and Connectivity Engineering
518-436-6433





From: Ed Gould <edgou...@COMCAST.NET>
To: IBM-...@LISTSERV.UA.EDU
Date: 06/27/2012 00:44
Subject: 'Inexperienced' RBS tech operative's blunder led to
banking meltdown
Sent by: IBM Mainframe Discussion List <IBM-...@LISTSERV.UA.EDU>



http://www.theregister.co.uk/2012/06/26/rbs_natwest_ca_technologies_outsourcing/



====================
This email/fax message is for the sole use of the intended
recipient(s) and may contain confidential and privileged information.
Any unauthorized review, use, disclosure or distribution of this
email/fax is prohibited. If you are not the intended recipient, please
destroy all paper and electronic copies of the original message.

Mark Zelden

unread,
Jun 27, 2012, 10:45:48 AM6/27/12
to
On Wed, 27 Jun 2012 09:49:03 -0400, Knutson, Sam <SKnu...@GEICO.COM> wrote:

<Big snip>

Well put Sam.

>
>The Register is fun reading sometimes but they rarely have enough detail to get real insight into what actually happened. So really anything we say here is just idle speculation.
>

Exactly what I told an old manager of mine who emailed me the
story after it started circulating at his office. I told him to think
of it as the "National Enquirer of IT". :-)

Mark
--
Mark Zelden - Zelden Consulting Services - z/OS, OS/390 and MVS
mailto:ma...@mzelden.com
Mark's MVS Utilities: http://www.mzelden.com/mvsutil.html
Systems Programming expert at http://expertanswercenter.techtarget.com/

J. Cassidy

unread,
Jun 27, 2012, 10:58:24 AM6/27/12
to
The Guardian has a version that is more or less readable, albeit meant for
the man on the street.

Some of the comments are rather "pungent".

http://www.guardian.co.uk/technology/2012/jun/25/how-natwest-it-meltdown?newsfeed=true

=> On Wed, 27 Jun 2012 09:49:03 -0400, Knutson, Sam <SKnu...@GEICO.COM>
=> wrote:
=>
=> <Big snip>
=>
=> Well put Sam.
=>
=>>
=>>The Register is fun reading sometimes but they rarely have enough detail
=>> to get real insight into what actually happened. So really anything we
=>> say here is just idle speculation.
=>>
=>
=> Exactly what I told an old manager of mine who emailed me the
=> story after it started circulating at his office. I told him to think
=> of it as the "National Enquirer of IT". :-)
=>
=> Mark
=> --
=> Mark Zelden - Zelden Consulting Services - z/OS, OS/390 and MVS
=> mailto:ma...@mzelden.com
=> Mark's MVS Utilities: http://www.mzelden.com/mvsutil.html
=> Systems Programming expert at http://expertanswercenter.techtarget.com/
=>
=> ----------------------------------------------------------------------
=> For IBM-MAIN subscribe / signoff / archive access instructions,
=> send email to list...@listserv.ua.edu with the message: INFO IBM-MAIN
=>


John Cassidy (Dipl.-Ingr.)

Kapellenstr. 21a

D-65193 Wiesbaden

EU



Mobile: +49 (0) 170 794 3616


http://www.JDCassidy.net

http://en.federaleurope.org/

http://sva-zhosting.com/en/index.php

Anne & Lynn Wheeler

unread,
Jun 27, 2012, 11:01:40 AM6/27/12
to
Thomas...@KEYBANK.COM (Tom Ambros) writes:
> "Complicated legacy mainframe system"? It's CA-7! If they think that's
> complicated they probably can't work a Mr. Coffee.

Key questions on the massive RBS / NatWest IT failure
http://www.zdnet.com/blog/projectfailures/key-questions-on-the-massive-rbs-natwest-it-failure/15706

references

RBS Bank joins the IT failures 'Hall of Shame'
http://www.zdnet.com/blog/projectfailures/rbs-bank-joins-the-it-failures-hall-of-shame/15685
Guardian's investigations suggest bank's problems began on Tuesday night
when it updated key piece of software called CA-7
http://www.guardian.co.uk/technology/2012/jun/25/how-natwest-it-meltdown

and

Update 6/27/12: According the The Register (a sensationalist tech news
site), an inexperienced computer operator in India caused the RBS
failure. Apparently, a routine problem arose during the upgrade
procedure, which is usually not a serious issue because administrators
can roll back to a previous and stable version of the software. In this
case, however, it seems the operator erroneously cleared the entire
transaction job queue, kicking off a long and difficult process of
reconstruction. The article adds: A complicated legacy
mainframe system at RBS and a team inexperienced in its quirks made the
problem harder to fix.

... snip ...

Inexperienced' RBS tech operative's blunder led to banking meltdown
http://www.theregister.co.uk/2012/06/26/rbs_natwest_ca_technologies_outsourcing/



--
virtualization experience starting Jan1968, online at home since Mar1970

McKown, John

unread,
Jun 27, 2012, 11:24:01 AM6/27/12
to
I somewhat agree (as difficult as that is for me). In today's world, it is expected that a worker will just walk in off the street and have the "intuitive" knowledge of how to use computers. As much as I dislike it personally, z/OS really needs the "new look" interface to present to the end users, and even programmers. I wonder if anybody has done a study of productivity between "old style" development using ISPF and edit-compile-test versus using the RD/z Eclipse based software.

--
John McKown
Systems Engineer IV
IT

Administrative Services Group

HealthMarkets®

9151 Boulevard 26 . N. Richland Hills . TX 76010
(817) 255-3225 phone .
john....@healthmarkets.com . www.HealthMarkets.com

Confidentiality Notice: This e-mail message may contain confidential or proprietary information. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. HealthMarkets® is the brand name for products underwritten and issued by the insurance subsidiaries of HealthMarkets, Inc. -The Chesapeake Life Insurance Company®, Mid-West National Life Insurance Company of TennesseeSM and The MEGA Life and Health Insurance Company.SM

Bill Fairchild

unread,
Jun 27, 2012, 11:31:21 AM6/27/12
to
I understand why all accumulated transactions have to be processed in the correct sequence if they are batched up daily, but why is an individual transaction not instantaneously and permanently processed with the results being committed right then? Why is it necessary to run huge batches of transactions at night? I am guessing that committing the transaction instantly would take too long for the average customer's patience when standing at an outdoors ATM in the rain. Are there any other reasons? When I buy an airplane ticket online, I get my whole transaction complete in one sitting and do not have to wait until the next day to receive confirmation of my reservation, but it always takes several minutes to do it online.

Bill Fairchild
Programmer
Rocket Software
408 Chamberlain Park Lane * Franklin, TN 37069-2526 * USA
t: +1.617.614.4503 * e: bfair...@rocketsoftware.com * w: www.rocketsoftware.com


-----Original Message-----
From: IBM Mainframe Discussion List [mailto:IBM-...@LISTSERV.UA.EDU] On Behalf Of J. Cassidy
Sent: Wednesday, June 27, 2012 9:58 AM
To: IBM-...@LISTSERV.UA.EDU
Subject: Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

Thomas Berg

unread,
Jun 27, 2012, 11:34:30 AM6/27/12
to
As I see it, the crucial point is not the faulting operators (?) competence but the (for this product) responsible staff as a whole.
If they had the typical old experienced and competent sysprogs/whatever in place they would never had let this be a possible outcome. They would in the first place have prohibited the possibility to do such an error, secondly make error obvious by any checks or signals and finally have a fast recovery procedure.

This as they by their experience would have imagined an error like this in advance.



Regards,
Thomas Berg
_______________________________________________________
Thomas Berg Specialist AM/SM&S SWEDBANK AB (publ)



> -----Ursprungligt meddelande-----
> Från: IBM Mainframe Discussion List [mailto:IBM-...@LISTSERV.UA.EDU]
> För McKown, John
> Skickat: den 27 juni 2012 17:24
> Till: IBM-...@LISTSERV.UA.EDU
> Ämne: Re: 'Inexperienced' RBS tech operative's blunder led to banking
> meltdown
>

Tom Ambros

unread,
Jun 27, 2012, 11:36:18 AM6/27/12
to
And whoever decided Tuesday night was the most appropriate probably has
some explaining to do. Right before a full sized cycle? Unless every
night is like that, probably not the most prudent decision.
This communication may contain privileged and/or confidential information. It is intended solely for the use of the addressee. If you are not the intended recipient, you are strictly prohibited from disclosing, copying, distributing or using any of this information. If you received this communication in error, please contact the sender immediately and destroy the material in its entirety, whether electronic or hard copy. This communication may contain nonpublic personal information about consumers subject to the restrictions of the Gramm-Leach-Bliley Act. You may not directly or indirectly reuse or redisclose such information for any purpose other than to provide the services for which you are receiving the information.

127 Public Square, Cleveland, OH 44114
If you prefer not to receive future e-mail offers for products or services from Key
send an e-mail to mailto:DNERe...@key.com with 'No Promotional E-mails' in the
SUBJECT line.

Phil Smith

unread,
Jun 27, 2012, 11:46:49 AM6/27/12
to
John McKown wrote:
>I somewhat agree (as difficult as that is for me). In today's world, it is expected that a worker will just walk in off the street and have the "intuitive" knowledge of how to use computers. As much as I dislike it personally, z/OS really needs the "new look" interface to present to the end users, and even programmers. I wonder if anybody has done a study of productivity between "old style" development using ISPF and edit-compile-test versus using the RD/z Eclipse based software.

Of course they have...many times. And they all show that IDEs are more productive, FVSO "more productive": programs get done faster. Not necessarily better, but faster. And it's a positive feedback loop: once the IDE is all anyone has used, old-school programming becomes unthinkable.

And that's how we get the instability that is Windows. Not even necessarily the Microsoft end of it: a near-infinite number of vendors with an infinite number of monkeys banging away at their IDEs, producing products that sort of work, but perhaps destabilize the underlying OS (or Office, or some other product) may be the real cause.

Of course this is an oversimplification, but when you make programming so simple, even a kid can do it, you get programs written by kids.

Even open source, for all its benefits, isn't a real solution: look at the plugin-container hack Firefox has resorted to. Is that strictly because Flash is so unstable? Chrome doesn't seem to have any significant Flash problems (or maybe I just use Firefox more). I'm on the Beta channel of Firefox right now because the production version was hanging my Windows 7 machines (maybe that's been fixed by now, but it *was* true). Worse, I had no real way to debug it.

OK, so I sound like a cranky old fart. If the shoe fits...
--
...phsiii

Mike Schwab

unread,
Jun 27, 2012, 11:47:12 AM6/27/12
to
Banking rules say you have to process the day's deposits before the
day's withdrawals.

Basically, you post the transaction to a transaction file which is
applied to the account after the cutoff time.

On Wed, Jun 27, 2012 at 10:30 AM, Bill Fairchild
<bfair...@rocketsoftware.com> wrote:
> I understand why all accumulated transactions have to be processed in the correct sequence if they are batched up daily, but why is an individual transaction not instantaneously and permanently processed with the results being committed right then?  Why is it necessary to run huge batches of transactions at night?   I am guessing that committing the transaction instantly would take too long for the average customer's patience when standing at an outdoors ATM in the rain.  Are there any other reasons?  When I buy an airplane ticket online, I get my whole transaction complete in one sitting and do not have to wait until the next day to receive confirmation of my reservation, but it always takes several minutes to do it online.
>
--
Mike A Schwab, Springfield IL USA
Where do Forest Rangers go to get away from it all?

Darth Keller

unread,
Jun 27, 2012, 11:47:59 AM6/27/12
to
>> And whoever decided Tuesday night was the most appropriate probably
has
>> some explaining to do. Right before a full sized cycle? Unless every

>> night is like that, probably not the most prudent decision.

>> Thomas Ambros

Agreed - I'd be curious to know how this got through any kind of
formalized Change Control Process.
ddk

This e-mail message and all attachments transmitted with it may
contain legally privileged and/or confidential information intended
solely for the use of the addressee(s). If the reader of this
message is not the intended recipient, you are hereby notified that
any reading, dissemination, distribution, copying, forwarding or
other use of this message or its attachments is strictly
prohibited. If you have received this message in error, please
notify the sender immediately and delete this message and all
copies and backups thereof. Thank you.

Rob Schramm

unread,
Jun 27, 2012, 11:50:14 AM6/27/12
to
http://cs.ua.edu/~SECSE08/Papers/Danis.pdf

Rob Schramm
Senior Systems Consultant
Imperium Group

Dave Day

unread,
Jun 27, 2012, 11:55:24 AM6/27/12
to
Bill,

Its more than just a specific transaction by an individual being
committed. A lot of it has to do with clearing house work. If you
write a check to someone, that someone will eventually deposit it in
their bank. That might not be the same bank that you use. It then has
to go thru a clearing house. The clearing house(s) used to work on a
time schedule, wherein all transactions for a preceding time period
would be batched up and processed, and distributed at a specific time on
a normal business day. Same thing holds true for payroll updates, gov't
checks, and a myriad of other types of transactions.

--Dave

Phil Smith

unread,
Jun 27, 2012, 11:55:49 AM6/27/12
to
Bill Fairchild wrote:
>I understand why all accumulated transactions have to be processed in the correct sequence if they are batched up daily, but why is an individual transaction not instantaneously and permanently processed with the results being committed right then? Why is it necessary to run huge batches of transactions at night? I am guessing that committing the transaction instantly would take too long for the average customer's patience when standing at an outdoors ATM in the rain. Are there any other reasons? When I buy an airplane ticket online, I get my whole transaction complete in one sitting and do not have to wait until the next day to receive confirmation of my reservation, but it always takes several minutes to do it online.

Besides/in addition to Mike Schwab's point, credit and signature debit transactions aren't processed in real-time: an authorization is processed, but the actual charge/debit doesn't go through until settlement.

<plug>Come to session 11409: The Payments Ecosystem: Security Challenges in the 21st Century<https://share.confex.com/share/119/webprogram/Session11409.html> at SHARE in Anaheim, Monday at 3PM, to learn more about how this</plug>
--
...phsiii

Dave Salt

unread,
Jun 27, 2012, 12:15:43 PM6/27/12
to
> From: John....@HEALTHMARKETS.COM
>I wonder if anybody has
done a study of productivity between "old style" development using ISPF
and edit-compile-test versus using the RD/z Eclipse based software.
>

Well if they did I hope they at least used SimpList or IPT instead of vanilla ISPF, otherwise they'd be painting the mainframe in a very unproductive and unfair light.

Dave Salt

SimpList(tm) - try it; you'll get it!

http://www.mackinney.com/products/program-development/simplist.html

Jerry Whitteridge

unread,
Jun 27, 2012, 12:35:31 PM6/27/12
to
I've just seen 3 HIPER PTF's from CA related to CA-7 that also fit the reported information on the RBS outage. I'd not be too hasty in blaming staff at this point.

Jerry Whitteridge
Lead Systems Programmer
Safeway Inc.
925 951 4184

If you feel in control
you just aren't going fast enough.
"Email Firewall" made the following annotations.
------------------------------------------------------------------------------

Warning:
All e-mail sent to this address will be received by the corporate e-mail system, and is subject to archival and review by someone other than the recipient. This e-mail may contain proprietary information and is intended only for the use of the intended recipient(s). If the reader of this message is not the intended recipient(s), you are notified that you have received this message in error and that any review, dissemination, distribution or copying of this message is strictly prohibited. If you have received this message in error, please notify the sender immediately.

==============================================================================

Bill Ashton

unread,
Jun 27, 2012, 12:39:15 PM6/27/12
to
Without having seen the HIPERs, it could be that they are trying to keep
others from blowing off their feet if they have staff who are
"operationally challenged."
B
--
Thank you and best regards,
*Billy Ashton*

Jerry Whitteridge

unread,
Jun 27, 2012, 12:47:03 PM6/27/12
to
One relates to a conditioning PTF required on the back level system to be able to backout an upgrade from a higher using a backup (DMPQ/MOVQ) process native to CA-7. As I read it they attempted to backout using the standard documented process but the downlevel system couldn't use the data formatted for the uplevel system and queue corruption might have occurred.

It's shutting the door after the event - but also points to a code not a process problem.

McKown, John

unread,
Jun 27, 2012, 1:15:14 PM6/27/12
to
Maybe. But all we have here is TSO/ISPF/SDSF along with some CompuWare products (Xpeditor and AbendAid mainly) to help us with development and testing. If "we" could get away with it, we'd likely get rid of ISPF/SDSF (not CompuWare, it is off maintenance and is perm licensed) to reduce costs. We look only at cost. Not value. Not productivity. Cost only. That may loosen up, depending of what SCOTUS says about Obamacare.

--
John McKown
Systems Engineer IV
IT

Administrative Services Group

HealthMarkets(r)

9151 Boulevard 26 * N. Richland Hills * TX 76010
(817) 255-3225 phone *
john....@healthmarkets.com * www.HealthMarkets.com

Confidentiality Notice: This e-mail message may contain confidential or proprietary information. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. HealthMarkets(r) is the brand name for products underwritten and issued by the insurance subsidiaries of HealthMarkets, Inc. -The Chesapeake Life Insurance Company(r), Mid-West National Life Insurance Company of TennesseeSM and The MEGA Life and Health Insurance Company.SM

> -----Original Message-----
> From: IBM Mainframe Discussion List
> [mailto:IBM-...@LISTSERV.UA.EDU] On Behalf Of Dave Salt
> Sent: Wednesday, June 27, 2012 11:16 AM
> To: IBM-...@LISTSERV.UA.EDU
> Subject: Re: 'Inexperienced' RBS tech operative's blunder led
> to banking meltdown
>

Dave Salt

unread,
Jun 27, 2012, 2:20:16 PM6/27/12
to
If a company has 100 programmers and they cost an average of $50 an hour, and they use a tool like SimpList or IPT to improve their productivity by 10%, they could get rid of 10 programmers or do 10% more work with the same number of programmers. Either way, that's a very REAL cost saving of ONE MILLION DOLLARS A YEAR. And if a company has 200 programmers they'd have a REAL cost saving of two million dollars a year (and so on).

Of course they'd have to deduct the cost of licensing the software ($8,000 for SimpList or ? for IPT), but it's still an ENORMOUS annual cost saving. And it's a very REAL cost saving; i.e. one that directly affects the bottom line. On top of that there's all the additional cost savings such as reduced training, reduced CPU, reduced storage, reduced printing, beating competitors to market, and much more (which by itself more than justifies the cost of the tool).

The fact that some PHB's can't comprehend this simple math is astonishing. Maybe they could 'save' even more money by getting rid of ISPF and telling all the programmers to use raw TSO? ;-)

Dave Salt

SimpList(tm) - try it; you'll get it!

http://www.mackinney.com/products/program-development/simplist.html




> Date: Wed, 27 Jun 2012 12:14:58 -0500
> From: John....@HEALTHMARKETS.COM
> Subject: Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown
> To: IBM-...@LISTSERV.UA.EDU

Frank Swarbrick

unread,
Jun 27, 2012, 2:35:11 PM6/27/12
to
I thought all "systems programmers" were perfect?




>________________________________
>Sent: Tuesday, June 26, 2012 10:37 PM
>Subject: 'Inexperienced' RBS tech operative's blunder led to banking meltdown
>
>http://www.theregister.co.uk/2012/06/26/rbs_natwest_ca_technologies_outsourcing/

Ed Finnell

unread,
Jun 27, 2012, 2:44:13 PM6/27/12
to
We used to subscribe to weekly world news, before times got tough. Then
there was the AA fiasco back in 80's where they managed to reply U to scratch
VTOC purge-1100 times!


In a message dated 6/27/2012 9:45:49 A.M. Central Daylight Time,
ma...@MZELDEN.COM writes:

I told him to think
of it as the "National Enquirer of IT". :-)



Mike Schwab

unread,
Jun 27, 2012, 3:18:38 PM6/27/12
to
On Wed, Jun 27, 2012 at 10:52 AM, Phil Smith <ph...@voltage.com> wrote:
> Bill Fairchild wrote:
>>I understand why all accumulated transactions have to be processed in the correct sequence if they are batched up daily, but why is an individual transaction not instantaneously and permanently processed with the results being committed right then?  Why is it necessary to run huge batches of transactions at night?   I am guessing that committing the transaction instantly would take too long for the average customer's patience when standing at an outdoors ATM in the rain.  Are there any other reasons?  When I buy an airplane ticket online, I get my whole transaction complete in one sitting and do not have to wait until the next day to receive confirmation of my reservation, but it always takes several minutes to do it online.
>
> Besides/in addition to Mike Schwab's point, credit and signature debit transactions aren't processed in real-time: an authorization is processed, but the actual charge/debit doesn't go through until settlement.
>
> <plug>Come to session 11409: The Payments Ecosystem: Security Challenges in the 21st Century<https://share.confex.com/share/119/webprogram/Session11409.html> at SHARE in Anaheim, Monday at 3PM, to learn more about how this</plug>
> --
> ...phsiii

Depositing checks by using a smart phone camera to electronic process
checks and image recognition by the bank is gaining traction.
Clearing the check image through the paying bank still takes time.
http://www.bankrate.com/financing/banking/more-banks-ok-smartphone-deposits/

Now, using a smart phone app to schedule an ATM withdraw instead of
using ATM cards and PINs hopes to end ATM card skimming and recording
the user entering the PIN.
http://www.bankrate.com/financing/banking/atm-withdrawals-without-a-card/?ec_id=m1078093

--
Mike A Schwab, Springfield IL USA
Where do Forest Rangers go to get away from it all?

Ed Gould

unread,
Jun 27, 2012, 3:53:46 PM6/27/12
to
Could it be instead that CA does not follow SMPE conventions (sup,pre
etc) ?

Ed

ps: I would love to see it that was the real reason. Only then CA
would be embarrassed (only after 10+ years)!
>> MA...@LISTSERV.UA.EDU>
>>
>>
>>
>>
>> http://www.theregister.co.uk/2012/06/26/
> --
> Thank you and best regards,
> *Billy Ashton*
>
> ----------------------------------------------------------------------
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to list...@listserv.ua.edu with the message: INFO IBM-MAIN
>
>
> "Email Firewall" made the following annotations.
> ----------------------------------------------------------------------
> --------
>
> Warning:
> All e-mail sent to this address will be received by the corporate e-
> mail system, and is subject to archival and review by someone other
> than the recipient. This e-mail may contain proprietary
> information and is intended only for the use of the intended
> recipient(s). If the reader of this message is not the intended
> recipient(s), you are notified that you have received this message
> in error and that any review, dissemination, distribution or
> copying of this message is strictly prohibited. If you have
> received this message in error, please notify the sender immediately.
>
> ======================================================================
> ========
>

McKown, John

unread,
Jun 27, 2012, 4:06:55 PM6/27/12
to
> -----Original Message-----
> From: IBM Mainframe Discussion List
> [mailto:IBM-...@LISTSERV.UA.EDU] On Behalf Of Dave Salt
> Sent: Wednesday, June 27, 2012 1:20 PM
> To: IBM-...@LISTSERV.UA.EDU
> Subject: Real cost savings (was: 'Inexperienced' RBS tech
> operative's blunder led to banking meltdown)
>
> If a company has 100 programmers and they cost an average of
> $50 an hour, and they use a tool like SimpList or IPT to
> improve their productivity by 10%, they could get rid of 10
> programmers or do 10% more work with the same number of
> programmers. Either way, that's a very REAL cost saving of
> ONE MILLION DOLLARS A YEAR. And if a company has 200
> programmers they'd have a REAL cost saving of two million
> dollars a year (and so on).

Hum, we have 4 programmers here. I don't know their real working hours. But I doubt we could do with less. People do insist on sleep and vacations.

>
> Of course they'd have to deduct the cost of licensing the
> software ($8,000 for SimpList or ? for IPT), but it's still
> an ENORMOUS annual cost saving. And it's a very REAL cost
> saving; i.e. one that directly affects the bottom line. On
> top of that there's all the additional cost savings such as
> reduced training, reduced CPU, reduced storage, reduced
> printing, beating competitors to market, and much more (which
> by itself more than justifies the cost of the tool).
>
> The fact that some PHB's can't comprehend this simple math is
> astonishing. Maybe they could 'save' even more money by
> getting rid of ISPF and telling all the programmers to use
> raw TSO? ;-)
>

Naw, as Marie Antoinette said: "Let them use vi!". You might be amazed (or disgusted, or sickened) at the stuff that I do on a z/OS UNIX shell prompt (via ssh, not TSO OMVS). I won't say more because: (1) I know it wouldn't work, and (2) last time I did, I got jumped all over for being a contemptible fool.

Or perhaps I could see if the RPF used under MVS by the Hercules/390 users would work on z/OS 1.12. At one time, I remember having a 3270 full screen editor called FSE which worked. And I'd see if there is a Q command for z/OS 1.12 (from CBT).

> Dave Salt

--
John McKown
Systems Engineer IV
IT

Administrative Services Group

HealthMarkets(r)

9151 Boulevard 26 * N. Richland Hills * TX 76010
(817) 255-3225 phone *
john....@healthmarkets.com * www.HealthMarkets.com

Confidentiality Notice: This e-mail message may contain confidential or proprietary information. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. HealthMarkets(r) is the brand name for products underwritten and issued by the insurance subsidiaries of HealthMarkets, Inc. -The Chesapeake Life Insurance Company(r), Mid-West National Life Insurance Company of TennesseeSM and The MEGA Life and Health Insurance Company.SM

Phil Smith

unread,
Jun 27, 2012, 4:33:19 PM6/27/12
to
Frank Swarbrick wrote:
>I thought all "systems programmers" were perfect?

Well, sum of us are.




















(Yes, that was deliberate!)

Scott Ford

unread,
Jun 27, 2012, 4:58:49 PM6/27/12
to
After reading the article a lot of issues...technical and non-technical....seems to be a epidemic of ppl who don't want to read ..seeing a lot of it lately. Plus good experienced people are hard to find.

Fwiw

Scott ford
www.identityforge.com

Henri zdevops Kuiper

unread,
Jun 27, 2012, 5:11:34 PM6/27/12
to
Maybe they need a free 2H zdevops session ;)

Sent from my wireless iPhone

Dave Salt

unread,
Jun 27, 2012, 5:13:34 PM6/27/12
to
> From: John....@HEALTHMARKETS.COM
> Hum, we have 4 programmers here.

Instead of programmers I should have said ISPF users. This includes developers, sysprogs, analysts, end-users, DBA's, help desk (etc). If this still means there's only 4 people at your company that would use SimpList or IPT then obviously the savings wouldn't be as high. But, it would still be worth it. If it costs $50 an hour to employee an ISPF user (which is a pretty low estimate considering salary and benefits and premises etc) then a 10% improvement in productivity would save the company about $10,000 per year per user. Given that the annual license fee for SimpList is only $8,000 (regardless of the number of users or number of LPAR's), this means even a single user would result in savings for the company.

Dave Salt

SimpList(tm) - try it; you'll get it!

http://www.mackinney.com/products/program-development/simplist.html




> Date: Wed, 27 Jun 2012 15:06:46 -0500
> From: John....@HEALTHMARKETS.COM
> Subject: Re: Real cost savings (was: 'Inexperienced' RBS tech operative's blunder led to banking meltdown)
> To: IBM-...@LISTSERV.UA.EDU

Scott Ford

unread,
Jun 27, 2012, 5:27:52 PM6/27/12
to
Phil,

Had a boss on VM man yrs ago, said learn the native VM and CMS commands first before you considering writing execs or clists. That was 20+ yrs ago, did that with VM , CMS, VTAM, TCPIP, etc ..if you understand how things work the GUIs are to me pretty but also makes people not think ....my $0.02 worth

Scott ford
www.identityforge.com

Scott Ford

unread,
Jun 27, 2012, 8:48:49 PM6/27/12
to
The tools are only as good as the people using them. Like a camera, one of my hobbies ,
People would see my Canon F-1 at time make a judgement call on how good he pictures at with type of camera. It also depends onto mind and eye of the photographer. I fel the same way about programmers and tools...

Scott ford
www.identityforge.com

glen herrmannsfeldt

unread,
Jun 27, 2012, 10:00:51 PM6/27/12
to
Scott Ford <scott_...@yahoo.com> wrote:
> The tools are only as good as the people using them.
> Like a camera, one of my hobbies ,
> People would see my Canon F-1 at time make a judgement
> call on how good he pictures at with type of camera.
> It also depends onto mind and eye of the photographer.
> I fel the same way about programmers and tools...

There is a group of photographers who like to test that
by showing how well they can do with cheap plastic
(and plastic lens) cameras. The camera that many years ago
cost a few dollars, then called Diana now called Holga,
now costs about $100. All due to supply and demand by this group.

There are contests where entries must be taken with a Holga
camera, or close enough (I don't know the exact rules).
It seems that among the features are light leaks from the
imperfect plastic molding, and otherwise low quality
plastic lens.

-- glen

David Crayford

unread,
Jun 28, 2012, 2:11:50 AM6/28/12
to
On 27/06/2012 11:23 PM, McKown, John wrote:
> I somewhat agree (as difficult as that is for me). In today's world, it is expected that a worker will just walk in off the street and have the "intuitive" knowledge of how to use computers. As much as I dislike it personally, z/OS really needs the "new look" interface to present to the end users, and even programmers. I wonder if anybody has done a study of productivity between "old style" development using ISPF and edit-compile-test versus using the RD/z Eclipse based software.

I can tell you from experience (I use RDz) that other than the actual
smart context assisting editors the tooling in RDz is generally poorly
implemented and less productive than using the ISPF equivalents.
Of course, I'm not talking about all the application web enabling stuff.
OTTOMH, FileManager, FaultAnalyzer, ApplicationPerformanceAnalyzer all
have plug-ins that are inferior to
the 3270 apps. It's a shame because they could have been much better.
Open a big dump in RDz from the spool system explorer, wait 15 minutes,
Eclipse chokes out an out-of-memory stack trace.
The mouse has to go down in history as possibly the worst invention for
hindering productivity. When I have to right-click and select from a
menu as apposed to tabbing and hitting a key then I generally
despise the UI and don't use it.

I also gave up on compiling in RDz. It's so much easier just to fire up
a shell or run JCL in a separate window. Even easier in Slickedit where
I can just use Putty and re-direct the output to my build window.
It's much easier to edit a Makefile than use a stupid dialog which has
unreasonable constraints.

> --
> John McKown
> Systems Engineer IV
> IT
>
> Administrative Services Group
>
> HealthMarkets®
>
> 9151 Boulevard 26 . N. Richland Hills . TX 76010
> (817) 255-3225 phone .
> john....@healthmarkets.com . www.HealthMarkets.com
>
> Confidentiality Notice: This e-mail message may contain confidential or proprietary information. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. HealthMarkets® is the brand name for products underwritten and issued by the insurance subsidiaries of HealthMarkets, Inc. -The Chesapeake Life Insurance Company®, Mid-West National Life Insurance Company of TennesseeSM and The MEGA Life and Health Insurance Company.SM
>
>
>
>> -----Original Message-----
>> From: IBM Mainframe Discussion List
>> [mailto:IBM-...@LISTSERV.UA.EDU] On Behalf Of Knutson, Sam
>> Sent: Wednesday, June 27, 2012 8:49 AM
>> To: IBM-...@LISTSERV.UA.EDU
>> Subject: Re: 'Inexperienced' RBS tech operative's blunder led
>> to banking meltdown
>>
>> -----Original Message-----
>> From: IBM Mainframe Discussion List
>> [mailto:IBM-...@LISTSERV.UA.EDU] On Behalf Of Tom Ambros
>> Sent: Wednesday, June 27, 2012 8:25 AM
>> To: IBM-...@LISTSERV.UA.EDU
>> Subject: Re: 'Inexperienced' RBS tech operative's blunder led
>> to banking meltdown
>>
>> "Complicated legacy mainframe system"? It's CA-7! If they
>> think that's complicated they probably can't work a Mr. Coffee.
>>
>> Thomas Ambros
>> Operating Systems and Connectivity Engineering
>> 518-436-6433
>>
>>
>>
>>
>>
>> From: Ed Gould <edgou...@COMCAST.NET>
>> To: IBM-...@LISTSERV.UA.EDU
>> Date: 06/27/2012 00:44
>> Subject: 'Inexperienced' RBS tech operative's blunder led to
>> banking meltdown
>> Sent by: IBM Mainframe Discussion List
>> <IBM-...@LISTSERV.UA.EDU>
>>
>>
>>
>> http://www.theregister.co.uk/2012/06/26/rbs_natwest_ca_technol
>> ogies_outsourcing/
>>
>>
>>
>> ====================
>> This email/fax message is for the sole use of the intended
>> recipient(s) and may contain confidential and privileged information.
>> Any unauthorized review, use, disclosure or distribution of this
>> email/fax is prohibited. If you are not the intended recipient, please
>> destroy all paper and electronic copies of the original message.

McKown, John

unread,
Jun 28, 2012, 8:44:36 AM6/28/12
to
Many thanks for that review. I tend to take the "reports on productivity" from "experts" with varying amounts of salt, depending on my experience with the reviewer. Some reviewers are given as much salt as anchovies.

My personal desire would be something akin to ISPF available from a z/OS UNIX shell (like from telnet or putty, not TSO). I don't know how ISPF is architected. That is, how dependant it is on TSO and 3270. I'd expect very dependant. But I would love it if it could use a "curses" or X terminal interface, and maybe the TSO dependant parts would work properly in an IKJTSOEV environment. However, I doubt that IBM will ever do this because they seem to be pushing RDz as the wave of the future. They may be correct in that. I have used the basic Eclipse on Linux/Intel and it is a nice IDE. For Java, I prefer Netbeans. Probably because I learned it before Eclipse existed. And for simple editing, I prefer vim or gvim. But I've also use kate, which is nice for multi-window editing.

--
John McKown
Systems Engineer IV
IT

Administrative Services Group

HealthMarkets(r)

9151 Boulevard 26 * N. Richland Hills * TX 76010
(817) 255-3225 phone *
john....@healthmarkets.com * www.HealthMarkets.com

Confidentiality Notice: This e-mail message may contain confidential or proprietary information. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. HealthMarkets(r) is the brand name for products underwritten and issued by the insurance subsidiaries of HealthMarkets, Inc. -The Chesapeake Life Insurance Company(r), Mid-West National Life Insurance Company of TennesseeSM and The MEGA Life and Health Insurance Company.SM

> -----Original Message-----
> From: IBM Mainframe Discussion List
> [mailto:IBM-...@LISTSERV.UA.EDU] On Behalf Of David Crayford
> Sent: Thursday, June 28, 2012 1:12 AM
> To: IBM-...@LISTSERV.UA.EDU
> Subject: Re: 'Inexperienced' RBS tech operative's blunder led
> to banking meltdown
>
> > HealthMarkets(r)
> >
> > 9151 Boulevard 26 . N. Richland Hills . TX 76010
> > (817) 255-3225 phone .
> > john....@healthmarkets.com . www.HealthMarkets.com
> >
> > Confidentiality Notice: This e-mail message may contain
> confidential or proprietary information. If you are not the
> intended recipient, please contact the sender by reply e-mail
> and destroy all copies of the original message.
> HealthMarkets(r) is the brand name for products underwritten
> and issued by the insurance subsidiaries of HealthMarkets,
> Inc. -The Chesapeake Life Insurance Company(r), Mid-West

David Crayford

unread,
Jun 28, 2012, 9:35:42 AM6/28/12
to
The concept of RDz is good. Give newbies an IDE which they're used to.
Unfortunately for seasoned mainframer's it just doesn't cut it. It's too
slow and click this, right click that etc is just stupid when compared
to a command line. It's the same concept as a *nix admin prefering the
shell to using a GUI. They get so much more value add from using a shell
to a GUI. Editors win every time. Check out www.slickedit.com for your
COBOL/HLASM to see what a mature editor can do. Compuware have licensed
it for their productivity suite. RDz does not come close.

On 28/06/2012 8:44 PM, McKown, John wrote:
> Many thanks for that review. I tend to take the "reports on productivity" from "experts" with varying amounts of salt, depending on my experience with the reviewer. Some reviewers are given as much salt as anchovies.
>
> My personal desire would be something akin to ISPF available from a z/OS UNIX shell (like from telnet or putty, not TSO). I don't know how ISPF is architected. That is, how dependant it is on TSO and 3270. I'd expect very dependant. But I would love it if it could use a "curses" or X terminal interface, and maybe the TSO dependant parts would work properly in an IKJTSOEV environment. However, I doubt that IBM will ever do this because they seem to be pushing RDz as the wave of the future. They may be correct in that. I have used the basic Eclipse on Linux/Intel and it is a nice IDE. For Java, I prefer Netbeans. Probably because I learned it before Eclipse existed. And for simple editing, I prefer vim or gvim. But I've also use kate, which is nice for multi-window editing.
>

McKown, John

unread,
Jun 28, 2012, 9:57:03 AM6/28/12
to
> -----Original Message-----
> From: IBM Mainframe Discussion List
> [mailto:IBM-...@LISTSERV.UA.EDU] On Behalf Of David Crayford
> Sent: Thursday, June 28, 2012 8:35 AM
> To: IBM-...@LISTSERV.UA.EDU
> Subject: Re: 'Inexperienced' RBS tech operative's blunder led
> to banking meltdown
>
> The concept of RDz is good. Give newbies an IDE which they're
> used to.
> Unfortunately for seasoned mainframer's it just doesn't cut
> it. It's too
> slow and click this, right click that etc is just stupid when
> compared
> to a command line. It's the same concept as a *nix admin
> prefering the
> shell to using a GUI. They get so much more value add from
> using a shell
> to a GUI. Editors win every time. Check out www.slickedit.com
> for your
> COBOL/HLASM to see what a mature editor can do. Compuware
> have licensed
> it for their productivity suite. RDz does not come close.

I'll look at it. Of course, I doubt the company would license it. I did a fast look and for a "Named User" (my personal copy) for Linux, the cost is $299.00 . Which makes it fairly expensive for me to buy for myself. The plus is that I can use it on multiple PCs, so long as only I use it and on one PC at a time. Which makes it perfect to be my personal editor, and keep it on a single flash drive which I take around with me.

--
John McKown
Systems Engineer IV
IT

Administrative Services Group

HealthMarkets(r)

9151 Boulevard 26 * N. Richland Hills * TX 76010
(817) 255-3225 phone *
john....@healthmarkets.com * www.HealthMarkets.com

Confidentiality Notice: This e-mail message may contain confidential or proprietary information. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. HealthMarkets(r) is the brand name for products underwritten and issued by the insurance subsidiaries of HealthMarkets, Inc. -The Chesapeake Life Insurance Company(r), Mid-West National Life Insurance Company of TennesseeSM and The MEGA Life and Health Insurance Company.SM

Dave Juraschek

unread,
Jun 28, 2012, 4:07:52 PM6/28/12
to
Mike said:
>Banking rules say you have to process the day's deposits before the
>day's withdrawals.
>Basically, you post the transaction to a transaction file which is
>applied to the account after the cutoff time.

No. This is a general accounting rule. (And actually, the rule is that transactions should be processed in the order received by an agent of the company/bank.)
It is a "bank", not a "banking" rule that you allude to.
I have personally experienced many banks (and subsequently moved my holdings to a new bank once I found this out) which apply withdrawls first, apply fines if this causes balance issues, and then apply deposits. I suggest that this is actually the "banking" rule used by most banks.
Worse yet, are the banks which hold deposits for days (most importantly including government checks and direct deposits or banking transfers which should be instantaneous, IMHO) solely so that they can gain interest on held client monies *before* they credit the deposit to their client - despite what effect that has on their client. (I know of several elderly folk who have been burned by very big, national banks who practice this regularly on SSI or Military payments from the government to them.)
The entire world wide banking system is corrupt and fully self serving. Look at how Iceland has reacted to this, actions taken and reforms enacted.

It's this client cavalier attitude of the banking industry that looks for cheap vs. quality solutions. From the articles cited thusfar, this debacle is a result of that mindset.

Kirk Talman

unread,
Jun 28, 2012, 4:23:01 PM6/28/12
to
IBM Mainframe Discussion List <IBM-...@LISTSERV.UA.EDU> wrote on
06/28/2012 04:07:46 PM:

> From: Dave Juraschek <david.j...@ASSOCIATES.DHS.GOV>
> Mike said:
> >Banking rules say you have to process the day's deposits before the
> >day's withdrawals.
> >Basically, you post the transaction to a transaction file which is
> >applied to the account after the cutoff time.
>
> No. This is a general accounting rule. (And actually, the rule is
> that transactions should be processed in the order received by an
> agent of the company/bank.)
> It is a "bank", not a "banking" rule that you allude to.

The rules vary from "bank" to "bank" and from country to country, and in
some cases the type of account. In the USA it used to be possible for
rules to vary from state to state, but I have not seen cases of that
lately.

There are also rules that vary in a similar way concerning the application
of fees and the calculation of interest.

The classes I have taken on this have caveats on each software option as
to where and how it may be used. Reminds me of standing in a waterfall.

Institutions subject to these rules, including service providers, have
departments called "compliance" whose job is to make sure rules are
followed and that advice given is accurate for the situation.

And as Dave indicated in the part of his message not copied, institutions
use differences in rules to competitive advantage.

-----------------------------------------
The information contained in this communication (including any
attachments hereto) is confidential and is intended solely for the
personal and confidential use of the individual or entity to whom
it is addressed. If the reader of this message is not the intended
recipient or an agent responsible for delivering it to the intended
recipient, you are hereby notified that you have received this
communication in error and that any review, dissemination, copying,
or unauthorized use of this information, or the taking of any
action in reliance on the contents of this information is strictly
prohibited. If you have received this communication in error,
please notify us immediately by e-mail, and delete the original
message. Thank you

Anne & Lynn Wheeler

unread,
Jun 28, 2012, 5:02:03 PM6/28/12
to
david.j...@ASSOCIATES.DHS.GOV (Dave Juraschek) writes:
> No. This is a general accounting rule. (And actually, the rule is
> that transactions should be processed in the order received by an
> agent of the company/bank.)
> It is a "bank", not a "banking" rule that you allude to.
> I have personally experienced many banks (and subsequently moved my
> holdings to a new bank once I found this out) which apply withdrawls
> first, apply fines if this causes balance issues, and then apply
> deposits. I suggest that this is actually the "banking" rule used by
> most banks.

ibm services has been outsourcing such processing for some number of
financial institutions ... there is folklore about trade-off between
having processing to always be under the limit where fine is applied
... versus cost savings on not enough processing for always handling
load spikes (offset by the sporadic fines; periodic claims that late
processing fines would wipe out any profit on the outsourcing contract)

--
virtualization experience starting Jan1968, online at home since Mar1970

Ed Gould

unread,
Jun 28, 2012, 7:22:43 PM6/28/12
to
On Jun 28, 2012, at 4:00 PM, Anne & Lynn Wheeler wrote:
>
> ibm services has been outsourcing such processing for some number of
> financial institutions ... there is folklore about trade-off between
> having processing to always be under the limit where fine is applied
> ... versus cost savings on not enough processing for always handling
> load spikes (offset by the sporadic fines; periodic claims that late
> processing fines would wipe out any profit on the outsourcing
> contract)
>

Lynn:

While this isn't about deposits/checks per se when I used to work at
a bank (albeit a long time ago) We had to shutdown the ATM's for one
hour every fall and incurred the fines to do so as we could not trust
the online application that was using a time stamp for purposes that
we couldn't control. I do not recall if it was CICS or a ryo system
but the time stamps were critical.

Ed

Scott Ford

unread,
Jun 29, 2012, 12:49:45 AM6/29/12
to
Ed,

I worked at a bank in manhattan and they had a encrypted line to the Fed , if down fines were $20000 a min..seen a lot ppl getting anxious over a down comm. line, but understandable

Scott ford
www.identityforge.com

Shmuel Metz , Seymour J.

unread,
Jul 1, 2012, 10:02:56 AM7/1/12
to
In <047067A6-A5C0-4F92...@comcast.net>, on 06/26/2012
at 11:37 PM, Ed Gould <edgou...@COMCAST.NET> said:

>Subject: 'Inexperienced' RBS tech operative's blunder led to banking
>meltdown

An *operator* was upgrading CA-7? Tell me it isn't true!

--
Shmuel (Seymour J.) Metz, SysProg and JOAT
Atid/2 <http://patriot.net/~shmuel>
We don't care. We don't have to care, we're Congress.
(S877: The Shut up and Eat Your spam act of 2003)

Ed Gould

unread,
Jul 1, 2012, 10:39:26 AM7/1/12
to
Shmuel:

Well if it was outsourced I wouldn't be surprised to much.
I suspect that these people sell themselves as experts in everything
CA-7 or MVS you name it. I have interviewed two foreign speaking
individuals and it turns out they read the manual and do not have a
clue beyond that.

Ed

Shmuel Metz , Seymour J.

unread,
Jul 1, 2012, 11:11:14 AM7/1/12
to
In <BLU168-W512A11E38...@phx.gbl>, on 06/27/2012
at 02:20 PM, Dave Salt <ds...@HOTMAIL.COM> said:

>The fact that some PHB's can't comprehend this simple math is
>astonishing.

Maybe. But maybe it's a case of the cost being in one budget and the
savings being in another. There's also the problem of people being
evaluated based on short term cash flow.

Be careful what you measure, because people will tailor their behavior
to what they're evaluated on.

--
Shmuel (Seymour J.) Metz, SysProg and JOAT
Atid/2 <http://patriot.net/~shmuel>
We don't care. We don't have to care, we're Congress.
(S877: The Shut up and Eat Your spam act of 2003)

Shmuel Metz , Seymour J.

unread,
Jul 1, 2012, 11:11:39 AM7/1/12
to
In <A6B9336CDB62BB46B9F87...@NRHMMS8P02.uicnrh.dom>,
on 06/28/2012
at 07:44 AM, "McKown, John" <John....@HEALTHMARKETS.COM> said:

>My personal desire would be something akin to ISPF available from a
>z/OS UNIX shell

If IBM were willing, the ISPF WSA would be a good start.

>and maybe the TSO dependant parts would work properly in an
>IKJTSOEV environment.

Why bother? Run the z/OS side in a batch job running IKJEFT01.

>And for simple editing, I prefer vim or gvim.

De gustibus. I prefer SPF for editing PC files. but at least you're
using vim and gvim rather than vi ;-)

--
Shmuel (Seymour J.) Metz, SysProg and JOAT
Atid/2 <http://patriot.net/~shmuel>
We don't care. We don't have to care, we're Congress.
(S877: The Shut up and Eat Your spam act of 2003)

zMan

unread,
Jul 1, 2012, 8:11:23 PM7/1/12
to
On Sun, Jul 1, 2012 at 10:33 AM, Ed Gould <edgou...@comcast.net> wrote:

> Shmuel:
>
> Well if it was outsourced I wouldn't be surprised to much.
> I suspect that these people sell themselves as experts in everything CA-7
> or MVS you name it. I have interviewed two foreign speaking individuals and
> it turns out they read the manual and do not have a clue beyond that.


And then they get on IBM-MAIN and ask incoherent questions...
--
zMan -- "I've got a mainframe and I'm not afraid to use it"

John Gilmore

unread,
Jul 1, 2012, 10:50:43 PM7/1/12
to
Regrettablly, incoherent questions are not limited to "foreign speakers".

Moreover, we have a number of "foreign speakers" among our most useful
contributors.

There was always the danger that it would do so, and this thread has
now taken a disagreeably xenophobic turn. I suggest that we kill it
off before it becomes even more offensive.

John Gilmore, Ashland, MA 01721 -USA

Anthony Thompson

unread,
Jul 1, 2012, 11:29:47 PM7/1/12
to
War story:

There was a certain major US-based mainframe software company (which I will not name) who decided to outsource the maintenance/development of one of their software products to an external company (not US-based, and once again, for fear of offending the overly-sensitive, I will not name the country). This product was partly written in some serious assembler code. Just before the contract was awarded, the in-house developers of the product thought it would be a good idea to do some due diligence, and interview the prospective future developers.

One of the questions they asked was: What is a supervisor call?

The answer: That's when we pick up the phone and call our boss.

When pressed, it became apparent they didn't know what a SVC was.

They didn't get the contract, despite it being very late in the process, it was awarded to the small company that I then worked for.

Unfortunately, some managers will take decisions based on their perceived financial value and little else. The risk analysis seems secondary, further, they don't seem to understand the value of decades of in-house expertise and knowledge. Such is business. Or, as Roger Waters (ex-Pink Floyd) said: "Don't be afraid, it's only business."

Ant.

Anne & Lynn Wheeler

unread,
Jul 1, 2012, 11:49:18 PM7/1/12
to
re:
http://www.garlic.com/~lynn/2012i.html#46 'Inexperienced' RBS tech operative's blunder led to banking meltdown
http://www.garlic.com/~lynn/2012i.html#58 'Inexperienced' RBS tech operative's blunder led to banking meltdown

it isn't RBS only problem:

Another Domino Falls in the LIBOR Banking Scam: Royal Bank of Scotland
http://www.rollingstone.com/politics/blogs/taibblog/another-domino-falls-in-the-libor-banking-scam-royal-bank-of-scotland-20120629ï

part of this:

A Huge Break in the LIBOR Banking Investigation
http://www.rollingstone.com/politics/blogs/taibblog/a-huge-break-in-the-libor-banking-investigation-20120628

Naked emperors, holy cows and Libor
http://www.atimes.com/atimes/Global_Economy/NF30Dj02.html

$700T worldwide:
http://www.atimes.com/atimes/Global_Economy/ML14Dj02.html

much of it in the US:
http://demonocracy.info/infographics/usa/derivatives/bank_exposure.html

a few more, now will anybody go to jail??

Banks face lawsuits worth billions over Libor scam
http://www.independent.co.uk/news/business/news/banks-face-lawsuits-worth-billions-over-libor-scam-7902918.html
U.K. Serious Fraud Office Opted Against Libor Criminal Probe: FT
http://www.bloomberg.com/news/2012-06-30/u-k-serious-fraud-office-opted-against-libor-criminal-probe-ft.html
Barclays LIBOR Market Manipulation Fraud To Boost Profits and Mask Insolvency, RBS, HSBC and Lloyds to Follow :: The Market Oracle ::
http://www.marketoracle.co.uk/Article35374.html
Banking scandal: how document trail reveals global scam
http://www.guardian.co.uk/business/2012/jun/30/banking-scandal-barclays-lawsuits-libor?newsfeed=trueï»

not directly ... other recent posts on derivatives
http://www.garlic.com/~lynn/2012h.html#45 Monopoly/ Cartons of Punch Cards
http://www.garlic.com/~lynn/2012i.html#17 Monopoly/ Cartons of Punch Cards
http://www.garlic.com/~lynn/2012i.html#29 Monopoly/ Cartons of Punch Cards

--
virtualization experience starting Jan1968, online at home since Mar1970

zMan

unread,
Jul 1, 2012, 11:56:49 PM7/1/12
to
On Sun, Jul 1, 2012 at 10:50 PM, John Gilmore <jwgl...@gmail.com> wrote:

> Regrettablly, incoherent questions are not limited to "foreign speakers".
>
> Moreover, we have a number of "foreign speakers" among our most useful
> contributors.
>
> There was always the danger that it would do so, and this thread has
> now taken a disagreeably xenophobic turn. I suggest that we kill it
> off before it becomes even more offensive.
>

Good point. I really wasn't thinking in those terms -- I was thinking of
the questions that are so basic that people usually ask, "Is this for a
school project?"

But yeah, it is at risk of turning into anti-foreigner rants. I'm out.

Scott Ford

unread,
Jul 2, 2012, 12:51:52 AM7/2/12
to
This industry is tough enough without a language barrier. Add the language barrier it gets down right ugly. Plus add cultural differences...man, plus lack of experience

Scott ford
www.identityforge.com

Mark Zelden

unread,
Jul 2, 2012, 8:54:52 AM7/2/12
to
On Sun, 1 Jul 2012 09:33:05 -0500, Ed Gould <edgou...@COMCAST.NET> wrote:

>Shmuel:
>
>Well if it was outsourced I wouldn't be surprised to much.
>I suspect that these people sell themselves as experts in everything
>CA-7 or MVS you name it. I have interviewed two foreign speaking
>individuals and it turns out they read the manual and do not have a
>clue beyond that.
>

Talk about a blanket generalization! Ed, when was the last time you
did any real work in MVS or worked with or in a company that
off shored any of their work?

And I don't know the specifics of this incident and haven't been following
it closely, but based on the CA-7 PTFs I saw released right afterwards
I would say the problem was related to the attempted upgrade (highly
doubtful that it was operators that attempted it) and backout coupled
with a CA-7 software issue that RBS couldn't have foreseen.

Mark
--
Mark Zelden - Zelden Consulting Services - z/OS, OS/390 and MVS
mailto:ma...@mzelden.com
Mark's MVS Utilities: http://www.mzelden.com/mvsutil.html
Systems Programming expert at http://expertanswercenter.techtarget.com/

McKown, John

unread,
Jul 2, 2012, 9:19:49 AM7/2/12
to
> -----Original Message-----
> From: IBM Mainframe Discussion List
> [mailto:IBM-...@LISTSERV.UA.EDU] On Behalf Of Mark Zelden
> Sent: Monday, July 02, 2012 7:55 AM
> To: IBM-...@LISTSERV.UA.EDU
> Subject: Re: 'Inexperienced' RBS tech operative's blunder led
> to banking meltdown
>
> On Sun, 1 Jul 2012 09:33:05 -0500, Ed Gould
> <edgou...@COMCAST.NET> wrote:
>
> >Shmuel:
> >
> >Well if it was outsourced I wouldn't be surprised to much.
> >I suspect that these people sell themselves as experts in everything
> >CA-7 or MVS you name it. I have interviewed two foreign speaking
> >individuals and it turns out they read the manual and do not have a
> >clue beyond that.
> >
>
> Talk about a blanket generalization! Ed, when was the last
> time you
> did any real work in MVS or worked with or in a company that
> off shored any of their work?
>
> And I don't know the specifics of this incident and haven't
> been following
> it closely, but based on the CA-7 PTFs I saw released right afterwards
> I would say the problem was related to the attempted upgrade (highly
> doubtful that it was operators that attempted it) and backout coupled
> with a CA-7 software issue that RBS couldn't have foreseen.
>
> Mark
> --
> Mark Zelden - Zelden Consulting Services - z/OS, OS/390 and MVS

I do know that we had a CA7 outage recently at IPL time, an S0C6. I am lucky, my manager is a CA7 expert and even knows some of the CA people personally. We did not have any corruption, just an ABEND. We just applied a HIPER to fix it. We were "bleeding edge" on CA7 maintenance.

--
John McKown
Systems Engineer IV
IT

Administrative Services Group

HealthMarkets(r)

9151 Boulevard 26 * N. Richland Hills * TX 76010
(817) 255-3225 phone *
john....@healthmarkets.com * www.HealthMarkets.com

Confidentiality Notice: This e-mail message may contain confidential or proprietary information. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. HealthMarkets(r) is the brand name for products underwritten and issued by the insurance subsidiaries of HealthMarkets, Inc. -The Chesapeake Life Insurance Company(r), Mid-West National Life Insurance Company of TennesseeSM and The MEGA Life and Health Insurance Company.SM

John Blythe Reid

unread,
Jul 2, 2012, 4:33:10 PM7/2/12
to
From what I've read, and it sounds quite as you would expect, the systems
programmers who applied the maintenance to CA-7 were based in the main
Edinburgh data centre whereas the team responsible for job scheduling using
CA-7 were based in Hyderabad. As we all know, when a critical job abends
during the middle of the night with a tight batch window, things become
quite stressful. And that rising stress level can lead to someone pressing
the wrong key. Normally the consequences are not as disastrous as this
one's been.

John.
--
John Blythe Reid,
Técnico de Sistemas de z/OS y de Sistemas Transaccionales,
Barcelona,
España.

John Gilmore

unread,
Jul 2, 2012, 5:33:39 PM7/2/12
to
John Reid has reminded us all of a generic weakness in the way we do
things. Sysprogs install new systems and maintain them, but they do
not usually test them. They may indeed have only the vaguest notions
of what some of them do.

Comprehensive IVPs have all but disappeared; but it it time, I think,
to reintroduce them. Moreover, there is a need for MVPs (Maintenance
Verification Procedures) too.

If they were always available sysprogs could properly be given the
responsibility for using them and exami ng their outputs.

John Gilmore, Ashland, MA 01721 - USA

Scott Ford

unread,
Jul 2, 2012, 6:58:25 PM7/2/12
to
John,

I don't know what sysprogs you have been around, but I can tell you i test, test, test , test...always have and always will. Part of the problem are the times and education and experience....
But I also think its how you were initially trained.

Scott ford
www.identityforge.com

John Gilmore

unread,
Jul 2, 2012, 7:43:51 PM7/2/12
to
'Sysprog' is unfortunately an ambiguous term. There are those who
build software for ISVs or, of course, IBM; and there are those who
keep a z/OS shop running.

As a part of an attempt to help in the selection of a new manager for
a systems programming group in a European insurance company I recently
spent 3 half days, one of them with each of three of the latter. They
installed things; did maintenance using SMP/E, etc., etc. They did
damn little testing, escept in the sense that the systematic
correction of errors in their own work is testing.

Moreover, it became clear that none of them knew or could know enough
about all of the components they were working with to test them
independently. (They were competent people, but they were not
polymaths.) This melancholy conclusion led me to the notion that IVPs
and MVPs, most of them necessarily supplied by vendors, needed to be
widely available.

These people have other needs too. They seem, for example, to have
very little sense of the ways in which notionally very different kinds
of IBM systems are almost all much alike in some ways. They cannot
make plausible, often (but not of course always) confirmed inferences
about how things work using this kind of knowledge. Moreover, I am
not sure I know how to communicate/teach these skills outside of
essentially one-on-one, apprenticeship situations.

John Gilmore, Ashland, MA 01721 - USA

Scott Ford

unread,
Jul 2, 2012, 8:46:52 PM7/2/12
to
John,

I also worked with Europeans, while living in Europe, also Mexico, including the Swiss. In my old age, 62 in August, I have concluded that the new IT world order, that's what my boss alls it, the solution to testing , is call the vendor. We get ton of calls, usually because ppl don't read the manuals. Many of them i feel haven't had the fortune of learning z/OS from the early days. I learned CICS many moons ago with the source assemblies and PLMs. Unfortunately, the have shall we say disappeared.

Scott ford
www.identityforge.com

Ed Gould

unread,
Jul 2, 2012, 11:32:10 PM7/2/12
to
Mark:

I spoke to the individuals that I have interviewed.

Ed

Shmuel Metz , Seymour J.

unread,
Jul 3, 2012, 8:09:08 AM7/3/12
to
In <24ED21E9-0EBC-42B6...@yahoo.com>, on 07/02/2012
at 12:51 AM, Scott Ford <scott_...@YAHOO.COM> said:

>This industry is tough enough without a language barrier. Add the
>language barrier it gets down right ugly.

I've worked with competent people with a language barrier. I've worked
with incompetent people who spoke flawless English. I'll take the
competent people with a language barrier, TYVM.

--
Shmuel (Seymour J.) Metz, SysProg and JOAT
Atid/2 <http://patriot.net/~shmuel>
We don't care. We don't have to care, we're Congress.
(S877: The Shut up and Eat Your spam act of 2003)

Shmuel Metz , Seymour J.

unread,
Jul 3, 2012, 8:09:23 AM7/3/12
to
In
<CAE1XxDHWTppHhJD1EthEUevNEy1EjJFsvxaATYw8VeDX=ef...@mail.gmail.com>,
on 07/01/2012
at 10:50 PM, John Gilmore <jwgl...@GMAIL.COM> said:

>Regrettablly, incoherent questions are not limited to "foreign
>speakers".

>Moreover, we have a number of "foreign speakers" among our most
>useful contributors.

Indeed, on both counts. No country has a monopoly on intelligence, and
no country has a monopoly on incompetence.

--
Shmuel (Seymour J.) Metz, SysProg and JOAT
Atid/2 <http://patriot.net/~shmuel>
We don't care. We don't have to care, we're Congress.
(S877: The Shut up and Eat Your spam act of 2003)

Shmuel Metz , Seymour J.

unread,
Jul 3, 2012, 8:11:58 AM7/3/12
to
In
<CAE1XxDEzxDZ4YJ3iJHU4TqXk34Cvp2J3ysWA0ese=1ScD...@mail.gmail.com>,
on 07/02/2012
at 05:33 PM, John Gilmore <jwgl...@GMAIL.COM> said:

>John Reid has reminded us all of a generic weakness in the way we

TINW.

>do things.

I don't know about you, but when I install a new system I test it,
have an SME test it or both.

>Comprehensive IVPs have all but disappeared;

That's a problem for a new system, but for an upgrade you should
already have a regression test.

--
Shmuel (Seymour J.) Metz, SysProg and JOAT
Atid/2 <http://patriot.net/~shmuel>
We don't care. We don't have to care, we're Congress.
(S877: The Shut up and Eat Your spam act of 2003)

Scott Ford

unread,
Jul 3, 2012, 8:44:34 AM7/3/12
to
Shmuel,

So have I ..competent people usually are on the same technical page, for sure

Scott ford
www.identityforge.com

Scott Ford

unread,
Jul 3, 2012, 11:04:57 AM7/3/12
to
Shmuel:
 
I work the same way, we have to test, no opinions, being a ISV. We support three security subsystems...RACF,ACF2 and Top-Secret. Even as a Sysprog on VM/VSE and MVS I always tested. To me personally, crazy not to, if I wanted to stay employed.
Secondly, I have a responsibility to my employer to do my jo2 to the best of my ability and then some...but thats me

Scott J Ford
Software Engineer
http://www.identityforge.com/
 


________________________________
From: Shmuel Metz (Seymour J.) <shmue...@PATRIOT.NET>
To: IBM-...@LISTSERV.UA.EDU
Sent: Tuesday, July 3, 2012 8:14 AM
Subject: Re: 'Inexperienced' RBS tech operative's blunder led to banking meltdown

0 new messages