Re: Check out Computer glitch to cause flight delays across U.S. - MarketWatch
Anne & Lynn Wheeler
lba...@BELLATLANTIC.NET (Alex UMX) writes:
> Well, 99.999% reliability means that 1/10000 of the time the thing is not
> reliable = about 8 hours per year.
> Not to confuse with z/VM reliability that was 99.999999 if I recall it correctly
> back in 1991 or so for VM/ESA 1.0.
about that period, we marketed ha/cmp for 1-800 against stratus. at the
time stratus took down system for software maintenance (would have
outage that precluded even five-nines). ha/cmp could have fall-over
between multiple servers as part of maint. strategy ... limiting outage
to few seconds. the response was that then stratus could install
multiple servers in (software fall-over) HA configuration ... to
eliminate maint. outage window. the response was then why spend for the
hardware redundancy
as an aside their box was also being relogo'ed and marketed as the S/88.
this recent post
http://www.garlic.com/~lynn/2009p.html#85 Anyone going to Supercomputers '09 in Portland
has some old "marketplace news" abstracts (29jun92) mentioning
remarketing branded stratus box ... the article might have a
misstatement ... the amount was what was paid for the program ... not
that much was actually sold.
as an aside ... i believe FAA has had some number of Flex-ES
installations (running mainframe software on intel platforms) ... old
post mentioning Flex-ES also available on stratus boxes
http://www.garlic.com/~lynn/2003d.html#10 Low-end processors (again)
for random other drift ... recent posts about old Jim Gray paper that
by early 80s, majority of outages had shifted from hardware faults to
other things
http://www.garlic.com/~lynn/2009.html#5 Is SUN going to become x86'ed ??
http://www.garlic.com/~lynn/2009.html#39 repeat after me: RAID != backup
http://www.garlic.com/~lynn/2009.html#47 repeat after me: RAID != backu
including scan of old copy of a version of presentation that I had
laying around ... and put up on the web:
http://www.garlic.com/~lynn/grayft84.pdf
I've mentioned before Jim and I had something of dust-up at acm sigops
in '91 about whether commodity components could be used for HA operation
... he was still at DEC and their party-line was vax/cluster. later when
he moved to m'soft ... he was up on stage for the announcement of m'soft
HA fallover product.
--
40+yrs virtualization experience (since Jan68), online at home since Mar1970
Anne & Lynn Wheeler
Anne & Lynn Wheeler <ly...@garlic.com> writes:
> for random other drift ... recent posts about old Jim Gray paper that
> by early 80s, majority of outages had shifted from hardware faults to
> other things
> http://www.garlic.com/~lynn/2009.html#5 Is SUN going to become x86'ed ??
> http://www.garlic.com/~lynn/2009.html#39 repeat after me: RAID != backup
> http://www.garlic.com/~lynn/2009.html#47 repeat after me: RAID != backu
>
> including scan of old copy of a version of presentation that I had
> laying around ... and put up on the web:
> http://www.garlic.com/~lynn/grayft84.pdf
re:
http://www.garlic.com/~lynn/2009q.html#26 Check out Computer glitch to cause flight delays across U.S. - MarketWatch
when I was out marketing ha/cmp,
http://www.garlic.com/~lynn/subtopic.html#hacmp
I coined the terms "disaster survivability" and "geographic surviability"
http://www.garlic.com/~lynn/submain.html#available
and I got asked to write a section in the corporate continuous
availability strategy document ... but it got pulled because both
rochester & pok objected (because at the time, they couldn't meet the
requirements).
after we left, we spent some time with large national financial
network. they claimed 100% availability for extended number of years was
because of
1) ims hot-standby
2) automated operator
... the ims hot-standby involved machines separated by geographic
distances (failures had shifted primarily to human mistakes and local
environmental conditions).
some of this was back to when my wife had been con'ed into doing a stint
in POK responsible for loosely-coupled architecture ... where she
created "peer-coupled shared data" architecture
http://www.garlic.com/~lynn/submain.html#shareddata
... which, except for ims hot-standby ... saw very little uptake until
sysplex. that and the battles with the communication group oper whether
SNA had to be used for loosely-coupled coordination, resulted in her not
staying long in the position.
Anne & Lynn Wheeler
> This is kind of off the topic, but related. Wasn't there a discussion
> on IBM-Main a couple years ago about the air traffic control system
> being run on old IBM 3081s? If I remember right, and my memory isn't
> as good as it used to be, it was just a few years ago that these old
> machines used to control some of the air traffic control. Have these
> been retired yet?
re:
http://www.garlic.com/~lynn/2009q.html#26 Check out Computer glitch to cause flight delays across U.S. - MarketWatch
http://www.garlic.com/~lynn/2009q.html#28 Check out Computer glitch to cause flight delays across U.S. - MarketWatch
we had project a decade ago with some of the people that had done the
1960s implementation ... and later left to form their own company to do
various things.
these were modified 360/50s that ran in triple configuration.
this comes up in the virtual machine folklore. the cambridge science
center was trying to get a 360/50 to build their hardware modifications
for supporting virtual memory ... but because all the available 360/50s
were going to air traffic control ... the science center had to settle
for 360/40. the first virtual machine system then was "cp/40" (instead
of "cp/50") ... morphing into cp67 (when they got a standard 360/67 that
came with hardware virtual memory support) ... which later morphed into
vm370. misc. past posts mentioning science center
http://www.garlic.com/~lynn/subtopic.html#545tech
some of that early 360 FAA software was moved along to various platforms
... including some of it eventually running in Flex-ES virtual machines
on intel platforms (possibly even stratus intel ... as mentioned in
previous post).
in the late 80s, we started ha/cmp product ... misc. past posts
http://www.garlic.com/~lynn/subtopic.html#hacmp
and part of that effort, we did detailed availability studies of tcp/ip
and various tcp/ip environments.
there have also been some number of (failed) FAA "modernization"
efforts. About the time we were doing ha/cmp ... there was such an
effort using triple-RS6000s as basic component. Because of our work on
high availability ... we were periodically asked to participate in some
of the discussions/reviews. There were some number of interesting
failure-modes that they overlooked ... and it was one of the
"modernization" efforts that ran into difficulty.
After we left in the 90s, we were invited in to consult with small
client/server startup that wanted to payment transactions ... the
startup had also invented this technology called "SSL" they wanted
to use.
Two of the people that we worked on ha/cmp for parallel Oracle
... referenced in jan92 meeting mentioned in this post
http://www.garlic.com/~lynn/95.html#13
were at this startup in charge of something called the "commerce
server". As part of doing what is now frequently called "electronic
commerce", we deployed something called the "payment gateway" ... misc.
past posts
http://www.garlic.com/~lynn/subnetwork.html#gateway
which acts as gateway for payment transactions between webservers on the
internet and payment infrastructure. An early prototype of this had a
situation where a merchant called up not being able to do transactions.
Normally, payment trouble-call desk has 5mins elapsed time to do first
level problem determination. In this case, three hours later, the
trouble ticket was closed with no-trouble-found.
In order to try and approach the non-internet availability ... we
deployed payment gateway on HA/CMP configuration with multiple diverse
routes ("telco provisioning") into different parts of the internet. we
also had to deploy/invent some number of compensating procedures to
compensate for vaguries of the internet ... as well as to compensate for
large number of identified security vulnerabilities.
One of the issues ... was that I had planned on also
broadcast/advertising the different routes ... but in the period of the
deployment; the internet backbone transitioned to hierarchical routing.
As a result, the remaining alternate path mechanism had to rely on
multiple-A record support in DNS (where the client is provided multiple
ip-address records in response to request domain lookup). The client
then cycles through each of the addresses until it finds one that makes
a succesful connection.
In any case, part of that deployment (for "electronic commerce")
included compensating for large number of different kinds of failures
that might happen anywhere in the internet infrastructure.
Anne & Lynn Wheeler
> efforts. About the time we were doing ha/cmp ... there was such an
> effort using triple-RS6000s as basic component. Because of our work on
> high availability ... we were periodically asked to participate in some
> of the discussions/reviews. There were some number of interesting
> failure-modes that they overlooked ... and it was one of the
> "modernization" efforts that ran into difficulty.
re:
http://www.garlic.com/~lynn/2009q.html#29 Check out Computer glitch to cause flight delays across U.S. - MarketWatch
a little ATC history:
http://www.centennialofflight.gov/essay/Government_Role/Air_traffic_control/POL15.htm
from above:
In December 1993, the FAA reviewed its order for the planned AAS. IBM
was far behind schedule and had major cost overruns. In 1994 the FAA
simplified its needs and picked new contractors. The revised
modernization program continued under various project names. Some
elements met further delays. In 1999, controllers began their first use
of an early version of the Standard Terminal Automation Replacement
System, which included new displays and capabilities for approach
control facilities. During the following year, FAA completed deployment
of the Display System Replacement, providing more efficient workstations
for en route controllers.
... snip ...
A little more historical (some of the documents are PDF)
http://www.faa.gov/about/history/
http://www.faa.gov/about/history/photo_album/air_traffic_control/index.cfm?cid=automation
Anne & Lynn Wheeler
Bi...@MAINSTAR.COM (Bill Fairchild) writes:
> 15. The agency accepts delivery of the upgraded equipment and begins
> training its people in how to use the newer technology. The training
> is supplied, of course, by the winning vendor. The training
> instructors are being charged to the government agency at the rate of
> $350 per hour. Four months later the equipment starts being gradually
> phased in. In the case of the FAA, there are about 20 places around
> the country known as Air Route Traffic Control Centers, each with its
> own computers, disk storage units, radar units, radar-processing
> computers, etc. The FAA decides to phase in this new equipment one
> ARTCC at a time, and allows 2 months to go by before attempting to
> begin using the new equipment at each different ARTCC. 2 months
> multiplied by 20 ARTCCs equals 40 months.
certain (federal) division of the company was well known for some of
those tactics. in the late 80s when I was doing high-speed datatransport
project ... the division claimed they had to be briefed on the project.
misc. past posts
http://www.garlic.com/~lynn/subnetwork.html#hsdt
I scheduled an all day presentation ... and something like 20-30 of
their people showup. A few weeks later, I get a bill ... they wanted to
charge at the going rate for listening to my presentation. I wouldn't
pay ... may counter-offer was I would bill them for the presentation
(that they had requested).
In the early 90s ... doing ha/cmp
http://www.garlic.com/~lynn/subtopic.html#hacmp
we would periodically drop into the division president's office ... who
we were acquated ... but also knew his technical assistant reasonably
well ... old email reference about division decided to make ha/cmp
cluster scaleup their strategic direction
http://www.garlic.com/~lynn/2006w.html#email911119
... of course this was before the cluster scaleup effort was transferred
and we were told we couldn't work on anything with more than four
processors. This time they didn't charge me for their time (in the
above referenced meeting).
In anycase, at the time, the president's technical assistant was
spending almost all their time trying to spearhead getting the
FAA effort back on track. related history note
http://www.garlic.com/~lynn/200q.html#31 Check out Computer glitch to cause flight delays across U.S. - MarketWatch
as to the cluster scaleup ... some old newsclipping from '92
http://www.garlic.com/~lynn/2001n.html#6000clusters1
http://www.garlic.com/~lynn/2001n.html#6000clusters2
other old email
http://www.garlic.com/~lynn/lhwemail.html#medusa