comp.arch has made itself a sitting duck for spam
Robert Myers
duck for spam, because the spam stays on top while people visit and
leave without posting.
Time to reverse the traditional snobbishness of this group.
Many thanks to the serious architects and other knowledgeable people
who have been so generous with their knowledge and time here. I think
it unlikely that people with something worthwhile to say will want to
post into a sea of spam.
Time to get off your hands, lose your pride, and say something.
Please, though, no cross-posts outside comp.arch.* ? kthx.
Robert
Anton Ertl
>A newsgroup that gets significant visits but few posts is a sitting
>duck for spam, because the spam stays on top while people visit and
>leave without posting.
You seem to use a strange newsreader. All those that I have used show
me each posting (spam or not) only once, unless I do something
special. So having few on-topic postings does not help spam, rather
on the contrary (because people unsubscribe groups with a low
signal/noise ratio).
That being said, my news server has >2700 Postings to comp.arch from
the last 100 days, and very little of that is spam. Maybe you have a
bad news server that does not use cleanfeed and ignores the cancels by
the spam-cancel-bots.
- anton
--
M. Anton Ertl Some things have to be seen to be believed
an...@mips.complang.tuwien.ac.at Most things have to be believed to be seen
http://www.complang.tuwien.ac.at/anton/home.html
Niels Jørgen Kruse
> A newsgroup that gets significant visits but few posts is a sitting
> duck for spam, because the spam stays on top while people visit and
> leave without posting.
Find yourself a newsserver with a clean feed. Google groups is not it.
--
Mvh./Regards, Niels J�rgen Kruse, Vanl�se, Denmark
MitchAlsup
thread on that is in any way related to computers and tosses out
anything that is not.
Mitch
Robert Myers
> Robert Myers <rbmyers...@gmail.com> wrote:
> > A newsgroup that gets significant visits but few posts is a sitting
> > duck for spam, because the spam stays on top while people visit and
> > leave without posting.
>
> Find yourself a newsserver with a clean feed. Google groups is not it.
An elitist answer worthy of the heady days for comp.arch now long
past.
The spam is a minor irritation for me, not worth any effort to fix.
I'll look for the good stuff, anyway.
My concern is that the good stuff will disappear.
Robert.
Eugene Miya
Robert Myers <rbmye...@gmail.com> wrote:
Sitting duck?
That's all numoderated news groups.
>On Jul 4, 4:38=A0am, nos...@ab-katrinedal.dk (Niels J=F8rgen Kruse) wrote:
>> Robert Myers <rbmyers...@gmail.com> wrote:
>> > A newsgroup that gets significant visits but few posts is a sitting
>> > duck for spam, because the spam stays on top while people visit and
>> > leave without posting.
>>
>> Find yourself a newsserver with a clean feed. Google groups is not it.
>
>An elitist answer worthy of the heady days for comp.arch now long
>past.
Don't try to sound blue collar. He's not giving an elitist answer.
Social pressure for better news feeds and better news readers pre-dated
Prodigy and AOL.
Comp.arch was once heady?
>The spam is a minor irritation for me, not worth any effort to fix.
You are leaving it to news admins who set up cancel bots.
>I'll look for the good stuff, anyway.
>
>My concern is that the good stuff will disappear.
You want to sit passively?
--
Looking for an H-912 (container).
Eugene Miya
Usenet attempts to balance S/N assuming the rationality of its users, in
particular posters. I moderate one of the other comp groups. I agreed
to do it as an experiment taking over from the somewhat overloaded 2nd
moderator (moderator #1 who started it as comp.hypercube, and I still
stay in touch over 2 decades and Steve has gone one with his life as
will in in a few years). Moderation is insufficient. It's true that
the delay is more than a few ADD individuals can stand.
To post in my group, you have to go through a Barracuda box which send
me a reputation report (I never approve), then the Baynesian filter fills
my mail box, and some stuff goes into a Spam file and another a
quesitonable Hold file. So far in the recent years 2 legit posts got
marked as false positives that I have seen, the 2nd even being a CFP
which I should have caught. I suspect that that Conference did fine.
I'm not a good enough architect to moderate a general comp arch type group.
But the general consensus of news admins is that posters deserve an
unmoderated alternative w/o going to the alt.* hierarchy.
Comp.arch is far from being say a comp.compilers or a comp.risks.
BTW those moderators get money (small stipends).
If spam is an issue, get/build better tools. That's the Usenet way.
Eugene Miya
Robert Myers <rbmye...@gmail.com> wrote:
>A newsgroup that gets significant visits but few posts is a sitting
>duck for spam, because the spam stays on top while people visit and
>leave without posting.
Visits?
Spam propagates to unmoderated groups.
>Time to reverse the traditional snobbishness of this group.
Spare us the working class hero stuff.
>Many thanks to the serious architects and other knowledgeable people
>who have been so generous with their knowledge and time here. I think
>it unlikely that people with something worthwhile to say will want to
>post into a sea of spam.
>
>Time to get off your hands, lose your pride, and say something.
Have you signed Non-disclosure?
>Please, though, no cross-posts outside comp.arch.* ? kthx.
Cross posting is a useful feature. It's a unique tool brilliantly
considered from the Unix file system by Ellis and Truscott.
You can cross post, just make certain that if you post a follow up:
1) reduce and don't attribute all past text.
2) Cut down on newsgroups in the Follow-up line. Edit headers as much
as editing text content.
3) Get (if necessary badger) others to do that.
The great thing about Usenet is that one has separate email options.
Robert Myers
> In article <2db2edaf-eb11-4ca0-9699-800becd4d...@p29g2000yqh.googlegroups.com>,Hasn't always been this way.
> >On Jul 4, 4:38=A0am, nos...@ab-katrinedal.dk (Niels J=F8rgen Kruse) wrote:
> >> Robert Myers <rbmyers...@gmail.com> wrote:
> >> > A newsgroup that gets significant visits but few posts is a sitting
> >> > duck for spam, because the spam stays on top while people visit and
> >> > leave without posting.
>
> >> Find yourself a newsserver with a clean feed. Google groups is not it.
>
> >An elitist answer worthy of the heady days for comp.arch now long
> >past.
>
> Don't try to sound blue collar. He's not giving an elitist answer.
> Social pressure for better news feeds and better news readers pre-dated
> Prodigy and AOL.
>
> Comp.arch was once heady?
>
Yes.
> >The spam is a minor irritation for me, not worth any effort to fix.
>
> You are leaving it to news admins who set up cancel bots.
>
You bet I am. The idea that I should pay a fee and have to cope with
another password just to read Usenet is ridiculous.
> >I'll look for the good stuff, anyway.
>
> >My concern is that the good stuff will disappear.
>
> You want to sit passively?
>
That is the first time anyone has ever accused me of being too little
involved. I do, in fact, report spam to Google, for all the good it
does.
Robert.
Robert Myers
>
> >Time to get off your hands, lose your pride, and say something.
>
> Have you signed Non-disclosure?
Do you know about I.F. Stone and his weekly? Of course you do. Not
only was he black-listed, he was also nearly deaf and so couldn't
participate in the cocktail circuit. Most doors were closed to him,
anyway.
Even so, he was the first journalist to expose the deliberate
fabrications that resulted in the Gulf of Tonkin resolution and the
disastrous expansion of the Viet Nam war. Entirely from publicly-
available documents.
No, I haven't signed any non-disclosure agreements recently, nor have
I had a security clearance for a long time. I don't need any of those
to calculate a bi-section bandwidth, nor to look at the performance of
our "super" computers on some kinds of calculations, nor even to find
critical admissions from the national laboratories as to the inherent
limitations of the kinds of computers that make up almost the entire
"Top" 500 list.
The state of software is a national disgrace. It isn't always
possible to separate the software issues from the hardware issues, and
I have really very little interest in wading through endless and
pointless discussions about programming style and languages. The
*only* place I know of where the issue has been addressed in any
reasonable fashion is avionics, and it's an issue that's not really on
the radar of comp.arch.embedded.
You take some of this personally. I wish you wouldn't.
Robert.
Eugene Miya
>Spam propagates to unmoderated groups.
>
>
>Have you signed Non-disclosure?
>
>>Please, though, no cross-posts outside comp.arch.* ? kthx.
>
>Cross posting is a useful feature. It's a unique tool brilliantly
>considered from the Unix file system by Ellis and Truscott.
>You can cross post, just make certain that if you post a follow up:
>1) reduce and don't attribute all past text.
>2) Cut down on newsgroups in the Follow-up line. Edit headers as much
>as editing text content.
>3) Get (if necessary badger) others to do that.
>
>The great thing about Usenet is that one has separate email options.
In the time since posting, a spam thought occurred to me, and among
other things I got a piece of spam to my news group which made
it past the barracuda (it was from Argentina).
The thought which occurred to me was that I attended IJCAI for the first
time ever in my career. I started working with one of the most AI
skeptical people in the business (Pierce). And I found more over time.
So IJCAI is the Intl. Joint Conf. on AI. The AI community does
something interesting in waffling conferences inside and outside the
USA to be international. And like many conferences there was a jobs board.
One of the firms hiring was Telefonica, the telephone company of Spain
and also various parts of Latin America and Central America. I previous
got a lot of spam for them, and I really seriously thought about
defacing their poster jobs ad with "SPAMMERS" over it.
Maybe next time.
I want to see "AI" come up with a perfect Spam filter.
Eugene Miya
Robert Myers <rbmye...@gmail.com> wrote:
I'm out of here for the weekend.
I missed my BP test. My fault.
>On Jul 31, 6:26=A0pm, eug...@cse.ucsc.edu (Eugene Miya) wrote:
>> In article <2db2edaf-eb11-4ca0-9699-800becd4d...@p29g2000yqh.googlegroups=
>.com>,
>> Robert Myers =A0<rbmyers...@gmail.com> wrote:
>> Sitting duck?
>> That's all unmoderated news groups.
>>
>Hasn't always been this way.
Mechanically it has.
Culturally, the net discovered commerce.
We would not make it through life without discovering ways around problems.
>> Comp.arch was once heady?
>>
>Yes.
No, I would disagree.
As impressive at net.arch and Usenet are and once were, comments by the
Haubens' book, views by Bill Gates and Al Gore, and brief tours and
lurking by profs like Alan Smith and John Hennessy, the Cray folk, etc.
It's largely low level people who have time to hang here. To think most
people doing heady things here is like listening to Barb in a.f.c. and
taking everything she says as word of God.
>> >The spam is a minor irritation for me, not worth any effort to fix.
>>
>> You are leaving it to news admins who set up cancel bots.
>>
>You bet I am. The idea that I should pay a fee and have to cope with
>another password just to read Usenet is ridiculous.
I don't like passwords and looked into biometrics in the 80s, but the
users are the ones to fix things here.
>> >I'll look for the good stuff, anyway.
>> >My concern is that the good stuff will disappear.
>>
>> You want to sit passively?
>>
>That is the first time anyone has ever accused me of being too little
>involved. I do, in fact, report spam to Google, for all the good it
>does.
It's the specific level of effort.
Anyone can post.
Nick posts.
Anyone can merely spout.
A few of us work to change or keep the same Usenet's configuration.
Tricks are to attempt other things.
I'll get to your other longer posts netx week.
Robert Myers
> As impressive at net.arch and Usenet are and once were, comments by the
> Haubens' book, views by Bill Gates and Al Gore, and brief tours and
> lurking by profs like Alan Smith and John Hennessy, the Cray folk, etc.
> It's largely low level people who have time to hang here. To think most
> people doing heady things here is like listening to Barb in a.f.c. and
> taking everything she says as word of God.
>
No need to worry. I can't stand Barb, and she can't stand me. It's
why I don't go near a.f.c.
As to the rest of it (egos, posturing, name-dropping), I wish you
wouldn't waste your time.
Robert.
nm...@cam.ac.uk
Robert Myers <rbmye...@gmail.com> wrote:
>
>The state of software is a national disgrace. It isn't always
>possible to separate the software issues from the hardware issues, and
>I have really very little interest in wading through endless and
>pointless discussions about programming style and languages. The
>*only* place I know of where the issue has been addressed in any
>reasonable fashion is avionics, and it's an issue that's not really on
>the radar of comp.arch.embedded.
No, it's not. It's an international disgrace!
And the really dispiriting thing about those endless and pointless
debates is that the most plausible solution does lie down that
route - just not in that direction.
Regards,
Nick Maclaren.
Robert Myers
> In article <662a19d7-f6e4-4562-8143-af9cdc8fa...@w6g2000yqw.googlegroups.com>,
> Robert Myers <rbmyers...@gmail.com> wrote:
>
>
>
> >The state of software is a national disgrace. It isn't always
> >possible to separate the software issues from the hardware issues, and
> >I have really very little interest in wading through endless and
> >pointless discussions about programming style and languages. The
> >*only* place I know of where the issue has been addressed in any
> >reasonable fashion is avionics, and it's an issue that's not really on
> >the radar of comp.arch.embedded.
>
> No, it's not. It's an international disgrace!
>
international arena. I suspect that not even the entire international
community can squander the level of financial resources that the U.S.
does and still make no apparent (or even negative) progress. Eugene
has pointed to one aspect of the real problem, which is that the
decision-makers with money to spend (or budget) are more concerned (or
perhaps exclusively concerned) with protecting their fiefdoms than
with pursuing any more laudable goal.
> And the really dispiriting thing about those endless and pointless
> debates is that the most plausible solution does lie down that
> route - just not in that direction.
>
Some of the smartest theoreticians in this area are on the payroll of
the world's largest software company. Now *that's* discouraging.
Robert.
nm...@cam.ac.uk
Robert Myers <rbmye...@gmail.com> wrote:
>>
>> >The state of software is a national disgrace. ...
>>
>> No, it's not. =A0It's an international disgrace!
>>
>I'm sure you're right, but I don't know where to point a finger in the
>international arena.
Just point it anywhere - there's almost certainly a guilty party at
the end of it!
> I suspect that not even the entire international
>community can squander the level of financial resources that the U.S.
>does and still make no apparent (or even negative) progress.
Don't bet on it. The USA isn't half as dominant as most people think
it is - while it isn't true that the emperor has no clothes, his
apparel is more of a fashion statement than a functional garment.
> Eugene
>has pointed to one aspect of the real problem, which is that the
>decision-makers with money to spend (or budget) are more concerned (or
>perhaps exclusively concerned) with protecting their fiefdoms than
>with pursuing any more laudable goal.
And you think that is a specifically USA phenomenon?
>> And the really dispiriting thing about those endless and pointless
>> debates is that the most plausible solution does lie down that
>> route - just not in that direction.
>>
>Some of the smartest theoreticians in this area are on the payroll of
>the world's largest software company. Now *that's* discouraging.
Not really. It's an engineering problem, not a theoretical one.
The theory has already been proven to work in practice.
Regards,
Nick Maclaren.
Robert Myers
> In article <95c36a4a-fd7f-41ba-887b-529ef2c17...@k19g2000yqn.googlegroups.com>,
> Robert Myers <rbmyers...@gmail.com> wrote:
>
>
> > I suspect that not even the entire international
> >community can squander the level of financial resources that the U.S.
> >does and still make no apparent (or even negative) progress.
>
> Don't bet on it. The USA isn't half as dominant as most people think
> it is - while it isn't true that the emperor has no clothes, his
> apparel is more of a fashion statement than a functional garment.
>
Rest assured. I know that the Internet is now ruled by programmers
from Eastern Europe. I'm not happy about it. If you are, your
resentment of the former colonies is getting out of hand.
> > Eugene
> >has pointed to one aspect of the real problem, which is that the
> >decision-makers with money to spend (or budget) are more concerned (or
> >perhaps exclusively concerned) with protecting their fiefdoms than
> >with pursuing any more laudable goal.
>
> And you think that is a specifically USA phenomenon?
>
I simply have no way of knowing. I'm familiar only with US empire
builders. They are a fearsome species.
> >> And the really dispiriting thing about those endless and pointless
> >> debates is that the most plausible solution does lie down that
> >> route - just not in that direction.
>
> >Some of the smartest theoreticians in this area are on the payroll of
> >the world's largest software company. Now *that's* discouraging.
>
> Not really. It's an engineering problem, not a theoretical one.
> The theory has already been proven to work in practice.
>
Meaning someone has demonstrated an industrial-strength Unix (or
equivalent) based on any of these theories?
Robert.
Andrew Reilly
> I suspect that not even the entire international
> community can squander the level of financial resources that the U.S.
> does and still make no apparent (or even negative) progress.
Just curious: what are you basing this assertion on?
What would progress look like?
How is it that things are bad at the moment?
Who is it that is not achieving the ends that they might reasonably
expect?
I'm not saying that you're wrong, but it seems to me that there's plenty
of interesting and beneficial stuff going on in the realm of software
development at the moment.
--
Andrew
Robert Myers
users.org> wrote:
> On Sat, 01 Aug 2009 12:33:49 -0700, Robert Myers wrote:
> > I suspect that not even the entire international
> > community can squander the level of financial resources that the U.S.
> > does and still make no apparent (or even negative) progress.
>
> Just curious: what are you basing this assertion on?
> What would progress look like?
> How is it that things are bad at the moment?
> Who is it that is not achieving the ends that they might reasonably
> expect?
>
mechnically.
> I'm not saying that you're wrong, but it seems to me that there's plenty
> of interesting and beneficial stuff going on in the realm of software
> development at the moment.
>
All I see is more eye-candy and a never-ending procession of patches.
Microsoft just released a patch to block a hack that would have
invalidated dozens of quick fixes they had issued over quite a long
period of time. Windows is on SSBN's (nuclear-missile carrying
submarines). Does this strike you as even acceptable?
Where's the progress? No one can tell for sure what software written
in c is *supposed* to do, never mind what it actually does. It's all
just complete craziness.
Tell me where you see progress. We have Ada. Almost no one uses it.
we have suites of software that are used in very limited circumstances
to check the formal correctness of software written in a specially-
constrained subset of Ada. Almost no one uses it.
The DoD gets an F for network security when audited. Personal
information like credit card numbers is compromised hundreds of
thousands of victims at a time.
It's not as if no one were aware of any of this.
You tell me. Where's the good stuff?
Even at a somewhat pedestrian level, I can operate Linux in some
unusually secure ways, but, at the moment, I can't view a fair amount
of web content because the widely-used flash is broken (again).
Read the Linux Kernel mailing list for an exchange between Linus and
the maintainer of a subsystem of a subsystem. Linus doesn't like the
maintainer's attitude toward application software (if it relies on
kernel behavior that isn't guaranteed, it's the application's
problem). Linus has the standard posturre: if there is important
legacy software that depends on undocumented behavior, it can't be
broken. That's all very well, but I don't know how, even in theory,
you can write an operating system or build chips that is always
backward-compatible with what are essentially bugs.
Need I go on? Where are the successes?
Robert.
Robert
>> What would progress look like?
>> How is it that things are bad at the moment?
>> Who is it that is not achieving the ends that they might reasonably
>> expect?
>>
> Software with known mathematical properties that can be checked
> mechnically.
The next few years should be interesting. Some random things that could
pay off in a big way:
1) IntentionalSoftware.com
2) Microsoft Singularity
a) code contracts,
b) provably correct ASM.
c) Phoenix compiler - preserves meta data from each phase of the compile
- plugin interface for custom checks and transforms.
- Decompiles C++ and MSIL, too.
3) Windows 7 instrumentation that reports back odd/slow combinations of
subsystem interactions.
A modifiable code generator, provable code, and real world tracing
to identify crashes and problems. Get those cooperating and
identifying problems, and you can write plugins for the compiler that check
for
similar patterns. Looks like a virtuous cycle!
This is not perfect. Maybe only 99% of the code is provable, and that
ignores the
hardware. This could eliminate whole classes of bugs. Developers can get
off
the treadmill..
Robert Myers
> A modifiable code generator, provable code, and real world tracing
> to identify crashes and problems. Get those cooperating and
> identifying problems, and you can write plugins for the compiler that check
> for
> similar patterns. Looks like a virtuous cycle!
That all sounds good. I hope it's real.
I hadn't even mentioned the times when my ridiculously overpowered
desktop stops to think for ridiculously long times--probably as
applications and the OS play bean bag with wait loops.
Robert.
nm...@cam.ac.uk
Robert Myers <rbmye...@gmail.com> wrote:
>
>> >> And the really dispiriting thing about those endless and pointless
>> >> debates is that the most plausible solution does lie down that
>> >> route - just not in that direction.
>>
>> >Some of the smartest theoreticians in this area are on the payroll of
>>
>> Not really. =A0It's an engineering problem, not a theoretical one.
>> The theory has already been proven to work in practice.
>>
>Meaning someone has demonstrated an industrial-strength Unix (or
>equivalent) based on any of these theories?
!!!! You can't get there starting from Unix (or existing programming
languages, as far as that aspect is concerned, with the partial
exception of Ada). The proof that I am referring to is the systems
and methodologies that never really took off - capability systems,
the Z methodology and so on.
Regards,
Nick Maclaren.
nm...@cam.ac.uk
Robert <rob...@nospam.com> wrote:
>>> Just curious: what are you basing this assertion on?
>>> What would progress look like?
>>> How is it that things are bad at the moment?
>>> Who is it that is not achieving the ends that they might reasonably
>>> expect?
>>>
>> Software with known mathematical properties that can be checked
>> mechnically.
>
>The next few years should be interesting. Some random things that could
>pay off in a big way:
>1) IntentionalSoftware.com
>2) Microsoft Singularity
> a) code contracts,
> b) provably correct ASM.
> c) Phoenix compiler - preserves meta data from each phase of the compile
> - plugin interface for custom checks and transforms.
> - Decompiles C++ and MSIL, too.
>3) Windows 7 instrumentation that reports back odd/slow combinations of
> subsystem interactions.
>
>A modifiable code generator, provable code, and real world tracing
>to identify crashes and problems. Get those cooperating and
>identifying problems, and you can write plugins for the compiler
>that check for similar patterns. Looks like a virtuous cycle!
NO chance!
Firstly, the specifications are miles from being sufficiently self
consistent (let alone unambiguous!) to allow provability, even in
theory. That is true for Fortran and C++, and redoubled in spades
for C, C++'s C aspects, POSIX and (as far as I can tell) .NET.
Secondly, the specifications are far too lax to allow any hope of
detecting most of the common programming errors that show up as
logical errors - indeed, ghastly languages such as Java and C99
require many important programming errors to NOT be diagnosed.
Thirdly, it has been known for 35+ years that doing any serious
automated analysis (it wasn't called 'program proving' then, as
it isn't really program proving) is intractable on languages not
designed for it. I.e. theoretically impossible and practically
unrealistic.
I could go on - there are other reasons why that pig won't fly - but
that should be enough to be getting on with.
Regards,
Nick Maclaren.
van...@vsta.org
>> Who is it that is not achieving the ends that they might reasonably
>> expect?
> Software with known mathematical properties that can be checked
> mechnically.
While I agree with most of your observations, it's not as clear to me
that we even know if this approach is the one which moves us forward.
Certainly, to date, it has born as little practical fruit as any other.
Intuitively, it *feels* like a step in the right direction. If we ever
make fundamental progress, my guess is that it's going to have some
attributes of languages which support formal semantics, but it'll be
overall quite a different approach. For instance, Side effects have a
place in the world, and I don't think monads are the whole answer.
Regards,
Andy Valencia
Robert Myers
I was asked, so I had to give an answer. As a practical matter, I'm
not sure about it, either.
It's something that seems like it should be doable to a greater extent
than it now is, and, so far as I am concerned, the negative
proposition is well-established: visual inspection and testing are not
adequate.
As to fixes of the kind advocated by Andy, we've seen the fate of the
NX bit.
Beyond that, my beliefs are almost religious: if you're not doing
something like establishing correctness or some similarly demonstrable
property, you are engaging in an elaborate exercise in self-deception.
I keep hoping that the boundary between hardware and software will
disappear, because I think the hardware guys have much better tools
and attitudes, whether they see themselves as wedded to formal methods
or not. The root disease, as far as I am concerned, is "engineering"
in an abstract world (software) that has no clearly-defined
relationship to a physically-realizable system. The consequence is
that the "rules" aren't really rules at all. Feynman's dictum was
that nature cannot be fooled. Unfortunately, a committee of software
experts can be fooled, and we've seen it over and over again.
Robert.
Andrew Reilly
> Firstly, the specifications are miles from being sufficiently self
> consistent (let alone unambiguous!) to allow provability, even in
> theory. That is true for Fortran and C++, and redoubled in spades for
> C, C++'s C aspects, POSIX and (as far as I can tell) .NET.
Scheme has a complete semantic spec, and I believe that some members of
the ML clan do too. I agree that being able to unambiguously reason
about code is vital to being able to get both correct *and* efficient
programs. All of the popular low-level languages are both over and under
specified, IMO.
> Secondly, the specifications are far too lax to allow any hope of
> detecting most of the common programming errors that show up as logical
> errors - indeed, ghastly languages such as Java and C99 require many
> important programming errors to NOT be diagnosed.
Yes. I've been horrified at some of the things built into the Java
numeric model, as I've read more about it. A great shame. (How can it
be useful to *not* be *able* to trap on signed integer arithmetic
overflows?)
> Thirdly, it has been known for 35+ years that doing any serious
> automated analysis (it wasn't called 'program proving' then, as it isn't
> really program proving) is intractable on languages not designed for it.
> I.e. theoretically impossible and practically unrealistic.
Apart from the very real problem of specification capture, real programs
run into halting-problem-esque limitations on any attempt at proof, very
quickly. That's one of the reasons why I'm convinced that a system of
run-time checks and good engineering practice for module design and
testing will always be essential.
Yes, all effort should be expended to make the most useful and helpful
programming tools that we can, but any expectation that large-scale
programming is ever going to be *easy* are very wrong-headed, IMO. Doing
it well requires training and effort, and that's never going to go away
(until singularity and the AI come, but I'm not counting those
chickens...)
Cheers,
--
Andrew
Andrew Reilly
> On Aug 1, 8:14 pm, Andrew Reilly <andrew-newsp...@areilly.bpc-
> users.org> wrote:
>> On Sat, 01 Aug 2009 12:33:49 -0700, Robert Myers wrote:
>> > I suspect that not even the entire international
>> > community can squander the level of financial resources that the U.S.
>> > does and still make no apparent (or even negative) progress.
>>
>> Just curious: what are you basing this assertion on? What would
>> progress look like?
>> How is it that things are bad at the moment? Who is it that is not
>> achieving the ends that they might reasonably expect?
>>
> Software with known mathematical properties that can be checked
> mechnically.
There's only so much that you *can* check mechanically, ahead of time.
Of course what can be proven should, and there's a whole pile of useful
tools available to that end, but what can't be proven should be tested-
for.
>> I'm not saying that you're wrong, but it seems to me that there's
>> plenty of interesting and beneficial stuff going on in the realm of
>> software development at the moment.
>>
> All I see is more eye-candy and a never-ending procession of patches.
Sure, there's a lot of old, broken code that's still in use. I don't
know what you mean about eye-candy, though. Fancy IDEs? That's not what
I'm talking about, although some of them do have some useful features.
> Windows is on SSBN's (nuclear-missile carrying
> submarines). Does this strike you as even acceptable?
No. However that can't be regarded as Microsoft's fault. At least, not
the Windows-NT architects and coders' fault. I don't understand how
something as intrinsically complicated and abjectly not designed-for-
purpose could ever have gotten that job. Well, I do understand, but it
has more to do with human nature and business issues than it does with
software or language design.
> Where's the progress? No one can tell for sure what software written in
> c is *supposed* to do, never mind what it actually does. It's all just
> complete craziness.
The progress is in languages like Scheme, Hascall, Clean, Ocaml/F#,
assorted and associated proof and contract/blame systems, functional data
structures, dynamic recompilation technologies (like the Javascript trace
compiler in Chrome) and parallel/concurrent memory allocators and rate-
aware garbage collection systems. Not all of these exist in the same
package, unfortunately, so I can't point at any one of them and say "go,
use that and be happy", but they're coming together...
> Tell me where you see progress. We have Ada. Almost no one uses it. we
> have suites of software that are used in very limited circumstances to
> check the formal correctness of software written in a specially-
> constrained subset of Ada. Almost no one uses it.
We've just gone through a bubble, where functionality on the street (or
at least in the advertising copy) trumped any level of correctness or
reliability. Up until a couple of years ago, the average computer non-
sophisticate was used to their software crashing on a daily basis, and
their computer needing constant rebooting. We've mostly got past that,
thankfully, but I still see my wife bursting into tears when her word
processor suddenly throws away her last several hours' work, or
mysteriously deciding to change all of the formatting in some inscrutable
way. That will eventually change too, now that word-processor feature-
lists are not a selling feature.
How fast does a word processor need to be? Not especially fast at all.
There is *no* good excuse for those and similar office products to be
designed with anything other than absolute correctness and reliability as
their first concern. And that'll happen. Might not involve Ada, but
there are plenty of other alternatives that won't let you step off the
end of an array.
> The DoD gets an F for network security when audited. Personal
> information like credit card numbers is compromised hundreds of
> thousands of victims at a time.
>
> It's not as if no one were aware of any of this.
Security isn't *just* a question of program correctness (although that
certainly helps). Security seems to be both hard and deep. I know that
I don't know enough about the security implications of the computer
systems that I look after. Who does? Information management and similar
policy-level issues are really orthogonal to questions of program
correctness and efficiency. I don't think that they're taught or studied
enough. I know that they weren't taught *at all* when I was at Uni.
> You tell me. Where's the good stuff?
See above. There's plenty around, if you care to look.
> Even at a somewhat pedestrian level, I can operate Linux in some
> unusually secure ways, but, at the moment, I can't view a fair amount of
> web content because the widely-used flash is broken (again).
Yes. My FreeBSD box doesn't do flash at all. Sometimes I regard that as
a benefit, but then I know that I can go and use a Mac to see those
pages, or just not bother. I have hopes that HTML-5 and SVG and
Javascript will "fix" that problem, but they're not very strong hopes.
> Read the Linux Kernel mailing list for an exchange between Linus and the
> maintainer of a subsystem of a subsystem. Linus doesn't like the
> maintainer's attitude toward application software (if it relies on
> kernel behavior that isn't guaranteed, it's the application's problem).
> Linus has the standard posturre: if there is important legacy software
> that depends on undocumented behavior, it can't be broken. That's all
> very well, but I don't know how, even in theory, you can write an
> operating system or build chips that is always backward-compatible with
> what are essentially bugs.
I like the way that the FreeBSD group handle that sort of problem (and
I'd be surprised if Linux below the distro level was much different, but
then I often am): new dot-0 releases are allowed to break any kind of
backward compatability, as long as a backwards-compatability schim
exists, that can optionally preserve the old behaviour, for old
binaries. The most recent of these schims are compiled-into the kernel
by default, but they need not be, and there is a (fairly recent) policy
of deliberate schedules of removing support for things deprecated for a
sufficient period of time.
> Need I go on? Where are the successes?
Even if you don't decide to use it yourself, download and have a look at
PLT-scheme, and read some of the guides and documentation. Then realize
that such a large, sophisticated and well-documented system is under
rapid development by a bare handful of academics. The benefits of a
language with well-defined semantics that has been designed to support
composability and contract-based testability.
Not every large software project is a failure. Those are just the ones
that you hear about in the press. Quite a bit of software engineering in-
the-large seems to be well managed and successful.
My Mac laptops are only ever rebooted when a sufficiently intrusive
kernel upgrade is pushed onto them. I can't remember the last time any
of the software that I use on a day-to-day basis crashed while I was
using it. No, I lie: the web browser on my phone crashes from time to
time, but that doesn't take the whole phone down: I can re-start it and
go on with what I was doing.
I don't think that the situation is as bleak as you're making it out to
be.
Cheers,
--
Andrew
Robert Myers
users.org> wrote:
> On Sat, 01 Aug 2009 20:02:06 -0700, Robert Myers wrote:
> > On Aug 1, 8:14 pm, Andrew Reilly <andrew-newsp...@areilly.bpc-
> > users.org> wrote:
> >> On Sat, 01 Aug 2009 12:33:49 -0700, Robert Myers wrote:
> >> > I suspect that not even the entire international
> >> > community can squander the level of financial resources that the U.S.
> >> > does and still make no apparent (or even negative) progress.
>
> >> Just curious: what are you basing this assertion on? What would
> >> progress look like?
> >> How is it that things are bad at the moment? Who is it that is not
> >> achieving the ends that they might reasonably expect?
>
> > Software with known mathematical properties that can be checked
> > mechnically.
>
> There's only so much that you *can* check mechanically, ahead of time.
> Of course what can be proven should, and there's a whole pile of useful
> tools available to that end, but what can't be proven should be tested-
> for.
>
should have more in common with mathematics than with advertising
copy, the change will be permanent and propagate through every level
of software "engineering."
> >> I'm not saying that you're wrong, but it seems to me that there's
> >> plenty of interesting and beneficial stuff going on in the realm of
> >> software development at the moment.
>
> > All I see is more eye-candy and a never-ending procession of patches.
>
> Sure, there's a lot of old, broken code that's still in use. I don't
> know what you mean about eye-candy, though. Fancy IDEs? That's not what
> I'm talking about, although some of them do have some useful features.
>
mmhmm. The "Aero" interface, so far as I can tell, is essentially
useless. Has anyone (Dvorak, say) pointed this out? Why should I be
required to have a top of the line graphics card to implement a
useless feature?
I'm actually pleased with Vista, on the whole, but that's because my
current desktop is ridiculously over-powered.
> > Windows is on SSBN's (nuclear-missile carrying
> > submarines). Does this strike you as even acceptable?
>
> No. However that can't be regarded as Microsoft's fault. At least, not
> the Windows-NT architects and coders' fault. I don't understand how
> something as intrinsically complicated and abjectly not designed-for-
> purpose could ever have gotten that job. Well, I do understand, but it
> has more to do with human nature and business issues than it does with
> software or language design.
>
It *can't* be described as Microsoft's fault? Eugene has told us here
how important it is not to insult Bill Gates because he can do the
industry so much good. Really? Why not tell the DoD that having
Windows on SSBN's is a *really* dumb idea? Is that less important
than a vaccine for malaria?
> > Where's the progress? No one can tell for sure what software written in
> > c is *supposed* to do, never mind what it actually does. It's all just
> > complete craziness.
>
> The progress is in languages like Scheme, Hascall, Clean, Ocaml/F#,
> assorted and associated proof and contract/blame systems, functional data
> structures, dynamic recompilation technologies (like the Javascript trace
> compiler in Chrome) and parallel/concurrent memory allocators and rate-
> aware garbage collection systems. Not all of these exist in the same
> package, unfortunately, so I can't point at any one of them and say "go,
> use that and be happy", but they're coming together...
>
No one uses them. They have zero impact.
> > Tell me where you see progress. We have Ada. Almost no one uses it. we
> > have suites of software that are used in very limited circumstances to
> > check the formal correctness of software written in a specially-
> > constrained subset of Ada. Almost no one uses it.
>
> We've just gone through a bubble, where functionality on the street (or
> at least in the advertising copy) trumped any level of correctness or
> reliability. Up until a couple of years ago, the average computer non-
> sophisticate was used to their software crashing on a daily basis, and
> their computer needing constant rebooting. We've mostly got past that,
> thankfully, but I still see my wife bursting into tears when her word
> processor suddenly throws away her last several hours' work, or
> mysteriously deciding to change all of the formatting in some inscrutable
> way. That will eventually change too, now that word-processor feature-
> lists are not a selling feature.
>
How long is this bubble going to last? Will it end before I die?
> How fast does a word processor need to be? Not especially fast at all.
> There is *no* good excuse for those and similar office products to be
> designed with anything other than absolute correctness and reliability as
> their first concern. And that'll happen. Might not involve Ada, but
> there are plenty of other alternatives that won't let you step off the
> end of an array.
>
I'm not wedded to ada.
> > The DoD gets an F for network security when audited. Personal
> > information like credit card numbers is compromised hundreds of
> > thousands of victims at a time.
>
> > It's not as if no one were aware of any of this.
>
> Security isn't *just* a question of program correctness (although that
> certainly helps). Security seems to be both hard and deep. I know that
> I don't know enough about the security implications of the computer
> systems that I look after. Who does? Information management and similar
> policy-level issues are really orthogonal to questions of program
> correctness and efficiency. I don't think that they're taught or studied
> enough. I know that they weren't taught *at all* when I was at Uni.
>
Once people understand that anything that *can* be fudged *will* be
fudged, we'll be on the right track. How do you know that something
can't be fudged? You prove it, that's how.
So long as we are relying on the word of people with big reputations,
we're dead. Politically, it doesn't work that way. If you can hide
behind billionaire Bill Gates, you're ok. I will stop just short of
pointing to participants who would demonstrate my maxim.
> > You tell me. Where's the good stuff?
>
> See above. There's plenty around, if you care to look.
>
And who actually uses it, Andrew?
> > Even at a somewhat pedestrian level, I can operate Linux in some
> > unusually secure ways, but, at the moment, I can't view a fair amount of
> > web content because the widely-used flash is broken (again).
>
> Yes. My FreeBSD box doesn't do flash at all. Sometimes I regard that as
> a benefit, but then I know that I can go and use a Mac to see those
> pages, or just not bother. I have hopes that HTML-5 and SVG and
> Javascript will "fix" that problem, but they're not very strong hopes.
>
I'm sorry, but I'd like to stay in touch with what's actually
happening, not what the snobs think are happening. If you're a snob,
you don't need flash. If you're not, you do.
> > Read the Linux Kernel mailing list for an exchange between Linus and the
> > maintainer of a subsystem of a subsystem. Linus doesn't like the
> > maintainer's attitude toward application software (if it relies on
> > kernel behavior that isn't guaranteed, it's the application's problem).
> > Linus has the standard posturre: if there is important legacy software
> > that depends on undocumented behavior, it can't be broken. That's all
> > very well, but I don't know how, even in theory, you can write an
> > operating system or build chips that is always backward-compatible with
> > what are essentially bugs.
>
> I like the way that the FreeBSD group handle that sort of problem (and
> I'd be surprised if Linux below the distro level was much different, but
> then I often am): new dot-0 releases are allowed to break any kind of
> backward compatability, as long as a backwards-compatability schim
> exists, that can optionally preserve the old behaviour, for old
> binaries. The most recent of these schims are compiled-into the kernel
> by default, but they need not be, and there is a (fairly recent) policy
> of deliberate schedules of removing support for things deprecated for a
> sufficient period of time.
>
This is over my pay grade. That's why Linus is famous, and I'm not.
> > Need I go on? Where are the successes?
>
> Even if you don't decide to use it yourself, download and have a look at
> PLT-scheme, and read some of the guides and documentation. Then realize
> that such a large, sophisticated and well-documented system is under
> rapid development by a bare handful of academics. The benefits of a
> language with well-defined semantics that has been designed to support
> composability and contract-based testability.
>
> Not every large software project is a failure. Those are just the ones
> that you hear about in the press. Quite a bit of software engineering in-
> the-large seems to be well managed and successful.
>
Tell that to the people who have had their identities stolen thanks to
the kind of self-congratulation you are announcing.
> My Mac laptops are only ever rebooted when a sufficiently intrusive
> kernel upgrade is pushed onto them. I can't remember the last time any
> of the software that I use on a day-to-day basis crashed while I was
> using it. No, I lie: the web browser on my phone crashes from time to
> time, but that doesn't take the whole phone down: I can re-start it and
> go on with what I was doing.
>
> I don't think that the situation is as bleak as you're making it out to
> be.
>
I actually think I'm soft-pedaling the reality so as not to be
dismissed as a kook. It's really bad. "All your network are belong
to us." I don't *think* this particular machine is compromised, but
how would I know?
Rpbert.
Andrew Reilly
> No one uses them. They have zero impact.
I really don't understand your argument, then. Of course no one (to
within experimental error) uses them. They're new. Look at them on
their merits, and how they might be of use to you, personally. Read some
of the associated papers. The similarity to mathematics (and the
concommitant maleability) is one of the things that strongly attracts me
to scheme, btw.
I don't understand how you get to describe as zero impact the compilation
technologies that are driving the web-delivered applicaiton change that
seems to be going on at the moment. Give the rest some time and some
analysis.
Or you can keep using the ball-of-string and gaffer-tape tools that you
and everyone else learned twenty or thirty years ago, and everything will
be exactly the same.
Re: the function bubble? It's finished. The Forrest Curve is one
aspect, people realizing that they don't use all of the functions (or
even know about most of them) in their application was the other. Once
you get to that point, having the thing not crash is more important to
you.
Regarding identity theft and credit card fraud, I think that's much more
a function of data retention and management policies, and organization-
level security processes, by and large, than a "software crisis". As I
said: security seems to be hard, and I certainly wasn't taught much about
it, so I don't know the answer. I very much doubt that any of it has to
do with "self congratulation" by me or anyone else. How is me being
impressed by someone elses' impressive piece of software engineering self
congratulation, anyway?
Cheers,
--
Andrew
Robert Myers
users.org> wrote:
>
> Regarding identity theft and credit card fraud, I think that's much more
> a function of data retention and management policies, and organization-
> level security processes, by and large, than a "software crisis". As I
> said: security seems to be hard, and I certainly wasn't taught much about
> it, so I don't know the answer. I very much doubt that any of it has to
> do with "self congratulation" by me or anyone else. How is me being
> impressed by someone elses' impressive piece of software engineering self
> congratulation, anyway?
>
What's an impressive piece of software, anyway, and how do you know
it's impressive?
Last 100 advisories:
http://www.packetstormsecurity.org/advisories100.html
Last 100 exploits:
http://www.packetstormsecurity.org/exploits100.html
There are, to be sure, lots of tools with which to fight back, but you
need to know about them, how to use them, and you actually need to use
them. For certain, you can't count on the people who write the
applications.
Robert.
nm...@cam.ac.uk
Andrew Reilly <andrew-...@areilly.bpc-users.org> wrote:
>On Sun, 02 Aug 2009 11:58:36 +0100, nmm1 wrote:
>
>> Firstly, the specifications are miles from being sufficiently self
>> consistent (let alone unambiguous!) to allow provability, even in
>> theory. That is true for Fortran and C++, and redoubled in spades for
>> C, C++'s C aspects, POSIX and (as far as I can tell) .NET.
>
>Scheme has a complete semantic spec, and I believe that some members of
>the ML clan do too. I agree that being able to unambiguously reason
>about code is vital to being able to get both correct *and* efficient
>programs. All of the popular low-level languages are both over and under
>specified, IMO.
I will try to look at Scheme, but I was really thinking about the
"general purpose" languages.
>> Thirdly, it has been known for 35+ years that doing any serious
>> automated analysis (it wasn't called 'program proving' then, as it isn't
>> really program proving) is intractable on languages not designed for it.
>> I.e. theoretically impossible and practically unrealistic.
>
>Apart from the very real problem of specification capture, real programs
>run into halting-problem-esque limitations on any attempt at proof, very
>quickly. That's one of the reasons why I'm convinced that a system of
>run-time checks and good engineering practice for module design and
>testing will always be essential.
Agreed.
Regards,
Nick Maclaren.
Bernd Paysan
> Regarding identity theft and credit card fraud, I think that's much more
> a function of data retention and management policies, and organization-
> level security processes, by and large, than a "software crisis". As I
> said: security seems to be hard, and I certainly wasn't taught much about
> it, so I don't know the answer.
Security is actually not that hard, but people are obviously undereducated
(you give an example, but the banks give a much better one). And it seems to
be widespread. A lot of possible identity theft have to do with badly
understood concepts. E.g. take the "lost my password question" you find very
often: This is a gaping security hole, which usually leads to losing both
the password and the secret question, when the answer is guessable (and it
is - if you ask people for their favorite pet, cat and dog will do quite
often ;-). Or take the lack of a PKI on the Internet - we have the expensive
root CAs in the browser mess (for SSL), we are only just now going to get a
PKI with DNSSEC, but for DNS only; we ought to have it for the routers (BGP)
as well, and so on.
And when even those geeks who build the Internet have troubles to get it
right, how are banks supposed to get it right? "Security" for a banker is a
piece of paper (called "certificate") which can be bought or sold, or, as
nowadays more common, can be bought and then over night loses all its value
and becomes unsellable junk ;-).
--
Bernd Paysan
"If you want it done right, you have to do it yourself"
http://www.jwdt.com/~paysan/
nm...@cam.ac.uk
Bernd Paysan <bernd....@gmx.de> wrote:
>
>Security is actually not that hard, but people are obviously undereducated
>(you give an example, but the banks give a much better one). And it seems to
>be widespread. A lot of possible identity theft have to do with badly
>understood concepts. E.g. take the "lost my password question" you find very
>often: This is a gaping security hole, which usually leads to losing both
>the password and the secret question, when the answer is guessable (and it
>is - if you ask people for their favorite pet, cat and dog will do quite
>often ;-). Or take the lack of a PKI on the Internet - we have the expensive
>root CAs in the browser mess (for SSL), we are only just now going to get a
>PKI with DNSSEC, but for DNS only; we ought to have it for the routers (BGP)
>as well, and so on.
Not to say that you are forced to disclose your 'security' information
to all and sundry by the methods used for authentication. Like that
stupid 3-digit addition to credit cards - you have to supply that for
almost every online transaction nowadays, so it adds nothing useful
to your bare number.
Regards,
Nick Maclaren.
Bernd Paysan
> Not to say that you are forced to disclose your 'security' information
> to all and sundry by the methods used for authentication. Like that
> stupid 3-digit addition to credit cards - you have to supply that for
> almost every online transaction nowadays, so it adds nothing useful
> to your bare number.
A similar stupid concept is that of OpenID or that you can use your Google
account on all the uncountable Google services which you often don't know
they belong to Google: You can use your identity from one service (say
Google Mail) to log into a number of other sites. Haven't people been warned
to disclose your account and password to anybody else than the service
provider where you want to log onto? Now it becomes the norm.
It would be all right if this was a certificate based login: You have your
private key on your PC, get it certified by any OpenID providers out there,
and then securely log into your accounts whereever they are. But it isn't,
it's password based. A simple proxy instead of the forwarding implementation
required for (more) secure authentication reveals your password. And the
fact that there is a proxy is unknown to the user - who doesn't study the
source code of the login page.
Note that secure online payment is not rocket science. It just requires a
slightly more complicated protocol. You first place your order. Now the shop
places a request for payment in your banking account. You log into your bank
account, and grant those requests - payment done (optional escrow accounts
could make that a full transaction - payment only on successful delivery).
For offline transactions, you still can put your three xxx on the bill to
authorize it.
Anne & Lynn Wheeler
Andrew Reilly <andrew-...@areilly.bpc-users.org> writes:
> Regarding identity theft and credit card fraud, I think that's much more
> a function of data retention and management policies, and organization-
> level security processes, by and large, than a "software crisis". As I
> said: security seems to be hard, and I certainly wasn't taught much about
> it, so I don't know the answer. I very much doubt that any of it has to
> do with "self congratulation" by me or anyone else. How is me being
> impressed by someone elses' impressive piece of software engineering self
> congratulation, anyway?
security can be hard, especially when it involves patch-work solutions
... possibly being applied in the wrong places and the wrong way
(resulting in a never ending ongoing effort)... as opposed to being
designed as part of the core infrastructure.
we were brought in to consult with a small client/server startup that
wanted to do payment transactions on their server and they had invented
this technology called *SSL* that they wanted to us; the result is now
frequently called "electronic commerce". as part of the "electronic
commerce" we had to do some detailed look at assumptions about how *SSL*
was being implemented and deployed as well as some of these new business
operations calling themselves Certification Authorities. Almost
immediately, secure SSL deployment assemptions were being violated.
A few posts about recent SSL problem news items:
http://www.garlic.com/~lynn/2009k.html#21 Security certificate warnings don't work, researchers say
http://www.garlic.com/~lynn/2009k.html#23 Security certificate warnings don't work, researchers say
http://www.garlic.com/~lynn/2009k.html#24 Barclays online banking borked
http://www.garlic.com/~lynn/2009k.html#25 Don't Take Fraud Out of Context
http://www.garlic.com/~lynn/2009k.html#28 Network Solutions breach exposed 500k card accounts
http://www.garlic.com/~lynn/2009k.html#38 More holes found in Web's SSL security protocol
http://www.garlic.com/~lynn/2009k.html#46 More holes found in Web's SSL security protocol
somewhat as a result, in the mid-90s we were asked to participate in the
x9a10 financial standards working group which had been given the
requirement to preserve the integrity of the financial infrastructure
for all retail payments (*ALL*, as in debit, credit, stored-value,
gift-card, ACH, point-of-sale, unattended, internet, transit turnstyle,
etc, i.e. *ALL*). the result was the x9.59 financial standard ... some
references
http://www.garlic.com/~lynn/x959.html#x959
as part of the effort we had to do some detailed, end-to-end, threat and
vulnerability assessments of the various environments. now x9.59 doesn't
do anything about skimming, evesdropping, harvesting, data breaches
(and/or the myriad other ways that information can leak out). what x9.59
did was slightly change the paradigm and make the information useless to
attackers for the purpose of fraudulent financial transactions. X9.59
didn't do anything about preventing the leaking of the information to
crooks ... it just eliminated the usefullness of that information to the
crooks for fraudlent transactions (the major form of identity theft that
usually has been making the press).
now the major use of *SSL* in the world today ... is this early stuff we
worked on called *electronic commerce* ... used to hide transactions
information as countermeasure to fraudulent financial transactions
... x9.59 prevents fraudulent transactions from any such information
leakage and therefore eliminates the major use of *SSL* currently.
we've used a couple metaphors in attempt to characterize the current
environment
* dual-use vulnerability metaphor
account number is required in a large number of different business
processes and is required to be readily available. at the same time
the account number has to be kept strictly confidential and never
divulged to anybody (not even those needing it for business processes,
since insiders have repeatedly been shown to be the major source of
identity theft). we've claimed that even if the planet was buried under
miles of information hiding encryption, that it wouldn't be sufficient
to prevent information leakage.
* security proportional to risk metaphor
to the merchant, knowledge of the account number is worth some percent
of the profit off the transaction (possibly a couple dollars, to the
transaction processor it may only be a few cents); that same knowledge
for the crook, is worth the account balance/credit-limit. as a result,
the crook may be able to outspend by a factor of 100 times attacking the
system (as the merchant or transaction processor can afford to spend
protecting the system).
* naked transaction metaphor
lots of archived blog activity & posts related to naked
transaction metaphor
http://www.garlic.com/~lynn/subintegrity.html#payments
...
some number of recent posts mentioning the above metaphors:
http://www.garlic.com/~lynn/2009b.html#13 US credit card payment house breaches by sniffing malware
http://www.garlic.com/~lynn/2009b.html#62 Study: Data breaches continue to get more costly for businesses
http://www.garlic.com/~lynn/2009f.html#36 PCI security rules may require reinforcements
http://www.garlic.com/~lynn/2009f.html#57 Data masking/data disguise Primer 1) WHY
http://www.garlic.com/~lynn/2009g.html#10 Top 10 Cybersecurity Threats for 2009, will they cause creation of highly-secure Corporate-wide Intranets?
http://www.garlic.com/~lynn/2009g.html#11 Top 10 Cybersecurity Threats for 2009, will they cause creation of highly-secure Corporate-wide Intranets?
http://www.garlic.com/~lynn/2009i.html#20 Online Banking's Innate Security Flaws
http://www.garlic.com/~lynn/2009j.html#11 Is anyone aware of a system that offers three layers of security and ID protection for online purchases or even over the counter POS purchases?
http://www.garlic.com/~lynn/2009j.html#13 PCI SSC Seeks Input on Security Standards
http://www.garlic.com/~lynn/2009j.html#41 How can we stop Credit card FRAUD?
--
40+yrs virtualization experience (since Jan68), online at home since Mar1970
Anne & Lynn Wheeler
> we've used a couple metaphors in attempt to characterize the current
> environment
>
> * dual-use vulnerability metaphor
re:
http://www.garlic.com/~lynn/2009k.html#54 The satate of software
i.e. majority of the current vulnerability is "replay attack" of
static data ... that same static data is also required for lots of
business processes ... setting up diametrically opposing requirements
... readily available for use by all the business processes .... and at
the same time required to be kept totally confidential and never
divulged for any reason.
the current security patch work efforts are attempting to hide the
static data ... even tho it exists at billions of different locations
around the world and is frequently being accessed for standard business
processes. as previously stated this is doomed to failure (i.e. even if
the planet was buried under miles of information hiding encryption there
would still be information leakage).
the x9.59 approach
http://www.garlic.com/~lynn/x959.html#x959
was to slightly tweak the paradigm and eliminate being able to use the
static data for "replay attacks" (totally ignoring whether the
information leaked or not ... in fact, assumed the information would be
guarenteed to leak and there was no way of stopping it).
as an aside, concurrently with the x9.59 activity ... there was some
efforts to specify a payment protocol using more traditional PKI and
digital certificates. Besides the fact that the traditional payment
transactions invalidated the basic premises for the use of PKI and
digital certificates there were several other significant issues.
shortly after one of these specifications was published, I did a
business operations profile as well as a public-key ops profile for the
specification. Then I got somebody that had taken the standard BSAFE
(public key) library to do benchmarks of the public-key ops profile on a
number of different platforms. I then reported the results back to some
of the people authoring the specification. Their response was that my
numbers were 100 times too slow ... however, in fact, the person doing
the benchmarks had made improvements to BSAFE library making it run four
times faster (so if they had ever done any public key work they should
have wondered why the benchmarks were four times faster). Much later
when there were pilot projects being deployed ... it turn out that the
benchmark numbers were within a couple percent of pilot measure numbers
(i.e. the BSAFE enhancements had been returned to the vendor).
Another issue was that typical PKI and digital certificates inflated the
standard payment transaction payload by a factor of 100 times ... so,
the straight-foward PKI approach not only added something like a
two-orders of magnitude processing bloat to payment transaction
processing ... but also added two-orders of magnitude payload size
bloat. misc. past posts on the subject:
http://www.garlic.com/~lynn/subpubkey.html#bloat
Since the X9A10 financial standard working group had been given the
requirement to preserve the integrity for *ALL* retail payments
(including very high value, very low value, very lightweight, etc)
... the work on X9.59 had to pay a lot of attention to both processing
and payload overheads as well as the security and integrity strength.
Part of it was that by the time we were through with applying *SSL* to
payment transactions ... it was clearly evident that PKI and digital
certificates were redundant and superfluous in that environment (as well
as representing enormous processing and payload bloat).
In the X9 financial standards group there was some realization regarding
the enormous PKI bloat/penalty in payment environment. There was one
effort to produce "compressed" digital certificates ... as trying to
partially mitigate the (rendundant and superfluous) PKI bloat/overhead.
There was work on producing a *compressed* digital certificate by
eliminating redundant fields (fields that would be the same in every
digital certificate that the entity's payment institution ... as relying
party would otherwise have).
We took it a step further ... by proving that an entity's payment
institution would already have every possible field normally carried by
a digital certificate ... in which case, digital certificates could be
compressed to zero bytes ... i.e. rather than claiming that it was
possible to perform PKI operations w/o the need of having (redundant and
superluous) digital certificates ... we prooved that it was possible to
have PKI operations with mandatory (zero-byte) digital certificates
appended to every transaction.
Robert Myers
>
> Security is actually not that hard, but people are obviously undereducated
> (you give an example, but the banks give a much better one). And it seems to
> be widespread.
Like, say, to notorious (but apparently undereducated) blackhat hacker
turned security consultant Kevin Mitnick, who just got tossed off his
provider because his site was successfully hacked too many times?
http://www.networkworld.com/news/2009/080309-mitnick-web-hosting.html?hpg1=bn
There is, of course, the story of Blue Frog.
I'm puzzled by the statements here. If we just add this widget or
that, of if everyone is a little more careful, everything will be ok.
Intel just recalled it's newest SSD's because of a firmware bug that
can result in the irretrievable loss of all data.
I'm thinking it would all go away if it were no longer possible to
sell hardware and software with an exclusion of remedies for
consequential damages.
Robert.
van...@vsta.org
>> Security is actually not that hard, but people are obviously undereducated
I would say security is just one aspect of reliability. My experience
is that reliability is really, really hard. Worthwhile, and hopefully
the future of software. But hard. And I really can't point to anybody
who's currently doing serious work on architecturally reliable systems
(proofs by counterexample welcome!).
Robert Myers <rbmye...@gmail.com> wrote:
> I'm thinking it would all go away if it were no longer possible to
> sell hardware and software with an exclusion of remedies for
> consequential damages.
I doubt we're going to fix this with lawyers. Offshore disposable
corporations, for instance, would still keep both prices and liability
low. Hopefully there's some way to map it to a business opportunity;
if I didn't already have a startup in the oven I'd probably look into
it.
Andy Valencia
Andy "Krazy" Glew
> I'm thinking it would all go away if it were no longer possible to
> sell hardware and software with an exclusion of remedies for
> consequential damages.
Amen!
--
The content of this message is my personal opinion only.
Although I am an employee - currently of Intel,
in the past of other computer companies such as AMD, Motorola, and Gould
- I reveal this only so that the reader may account
for any possible bias I may have towards my employer's products.
The statements I make here in no way represent my employer's position,
nor am I authorized to speak on behalf of my employer.
In fact, this posting may not even represent my personal opinion,
since occasionally I play devil's advocate.
Robert Myers
> Robert Myers <rbmyers...@gmail.com> wrote:
> > I'm thinking it would all go away if it were no longer possible to
> > sell hardware and software with an exclusion of remedies for
> > consequential damages.
>
> I doubt we're going to fix this with lawyers. Offshore disposable
> corporations, for instance, would still keep both prices and liability
> low. Hopefully there's some way to map it to a business opportunity;
> if I didn't already have a startup in the oven I'd probably look into
> it.
>
I can think of business models that would create incentives for
sellers to produce more reliable products without the necessary
involvement of remedies at law.
The question is: how much would anyone be willing to pay? It would
have to be enough to justify the effort, and everyone has become
accustomed to prices that won't pay for much.
Robert.
nm...@cam.ac.uk
>> On Aug 3, 5:03?am, Bernd Paysan <bernd.pay...@gmx.de> wrote:
>>> Security is actually not that hard, but people are obviously undereducated
>
>I would say security is just one aspect of reliability. My experience
>is that reliability is really, really hard. Worthwhile, and hopefully
>the future of software. But hard. And I really can't point to anybody
>who's currently doing serious work on architecturally reliable systems
>(proofs by counterexample welcome!).
I agree. And so does our local security expert. Without reliability,
there are always loopholes in the security ....
>Robert Myers <rbmye...@gmail.com> wrote:
>> I'm thinking it would all go away if it were no longer possible to
>> sell hardware and software with an exclusion of remedies for
>> consequential damages.
>
>I doubt we're going to fix this with lawyers. Offshore disposable
>corporations, for instance, would still keep both prices and liability
>low. Hopefully there's some way to map it to a business opportunity;
>if I didn't already have a startup in the oven I'd probably look into
>it.
It would lead to be bankruptcy of all small businesses and probably
the elimination of open source - yes, that has been the effect when
you become liable even for things you give away. Only the companies
to big and nasty to sue would remain, and security would NOT improve.
I am thinking about it for when I retire, but no way would I try
to start up a company - I know my limits!
Regards,
Nick Maclaren.
Anne & Lynn Wheeler
van...@vsta.org writes:
> I would say security is just one aspect of reliability. My experience
> is that reliability is really, really hard. Worthwhile, and hopefully
> the future of software. But hard. And I really can't point to anybody
> who's currently doing serious work on architecturally reliable systems
> (proofs by counterexample welcome!).
re:
http://www.garlic.com/~lynn/2009k.html#54 The satate of software
http://www.garlic.com/~lynn/2009k.html#60 The satate of software
reference meeting regarding doing cluster scaleup for ha/cmp product
http://www.garlic.com/~lynn/95.html#13
now two of the people in the above meeting (jan92), later left and show
up at a small client/server startup, responsible for something called
"commerce server" and we were brought in to consult because the startup
wanted to do payment transactions on their server (the startup had also
invented this technology called "SSL" they wanted to use).
in the ha/cmp part ... we were looking at requirements of five nines or
better ... and we view "security" as just one aspect of RAS
(reliability, availability, service). An example of "S" aspect ... we
went into one environment with 5-nines requirement, that had been using
a system purely based on hardware fault tolerant implementation ... but
required something like annual software upgrade. However, their outage
for single annual software maint. blew nearly a century's outage budget.
some old email mentioning the cluster scaleup part of the project
http://www.garlic.com/~lynn/lhwemail.html#medusa
shortly after the jan92 meeting, the cluster scaleup part of the project
was transferred, announced as a supercomputer, and we were told we
couldn't work on anything with more than four processor.
for slightly other drift:
http://www.garlic.com/~lynn/2008p.html#27 Father Of Financial Dataprocessing
mentions working with somebody that developed formal definition for
transaction properties ... which then made a difference for financial
auditors accepting computer based operations.
semi-related posts regarding the original sql/relational implementation
http://www.garlic.com/~lynn/submain.html#systemr
Eugene Miya
Robert Myers <rbmye...@gmail.com> wrote:
>On Aug 2, 9:20=A0pm, Andrew Reilly <andrew-newsp...@areilly.bpc->> > On Aug 1, 8:14=A0pm, Andrew Reilly <andrew-newsp...@areilly.bpc->> >> > =A0I suspect that not even the entire international
>> >> > community can squander the level of financial resources that the U.S.
>> >> > does and still make no apparent (or even negative) progress.
>>
>>
>should have more in common with mathematics than with advertising
>copy, the change will be permanent and propagate through every level
>of software "engineering."
Math isn't good enough.
Put some money into human factors research.
Logic, in and of itself, is insufficient to solve software's problems.
>> > Windows is on SSBN's (nuclear-missile carrying
>> > submarines). Does this strike you as even acceptable?
>>
>> No. However that can't be regarded as Microsoft's fault. At least, not
>> the Windows-NT architects and coders' fault. =A0I don't understand how
>> something as intrinsically complicated and abjectly not designed-for-
>> purpose could ever have gotten that job. =A0Well, I do understand, but it
>> has more to do with human nature and business issues than it does with
>> software or language design.
>>
>It *can't* be described as Microsoft's fault? Eugene has told us here
>how important it is not to insult Bill Gates because he can do the
>industry so much good.
I didn't say that.
First, I'm just trying to catch up getting back 2 weeks ago from Redmond
and also having East Coast visitors.
2nd, what's MS got to do with Gates? Gates left. Microsoft now is
different from my last trip there. This is Ballmer's MS. It's no
nonsense. Bing is the new thing. In Seattle, MS can do no wrong.
Boeing has gone to Chicago. Boeing is only a concern for the burbs of
Kent, Renton, and Everett, and lesser burbs. MS was well represented at
OSCON.
3rd: about Gates. The power brokers of the USA don't really care about him.
To them, he's just a nerd who has been whipped into place. Likely they
think they can manipulate him through Warren Buffett (a guess).
>Really? Why not tell the DoD that having
>Windows on SSBN's is a *really* dumb idea?
I saw NT (2000) on the USS Hopper on its commissioning as well as some
Aegis cruiser (the Little Rock maybe?). I suspect that certain parts of
the DoD alread know this is a dumb idea, but they are powerless to
develop a new OS much less update Ada. Not do much to influence
hardware development. Don't forget all the laptops used by the heads of
the US Administrator.
Most fire control systems don't use MS as its not cleared for real-time
applications. The space station uses Windows (and Ada).
Different parts of the DoD have responsibility for software which come
in contact with their people in various ways including Obama's fingers
(not just his secure version of the Blackbery).
The problem is that it is insufficient to merely say something is a dumb
idea. You have to have a completely formed alternative and a transition
plan to get there. Think Metric.
>Is that less important than a vaccine for malaria?
Are we going to see detonations?
>> > Where's the progress? =A0No one can tell for sure what software written=
> in
>> > c is *supposed* to do, never mind what it actually does. =A0It's all ju=
>st
>> > complete craziness.
>>
>> The progress is in languages like Scheme, Hascall, Clean, Ocaml/F#,
Haskell.
>I actually think I'm soft-pedaling the reality so as not to be
>dismissed as a kook. It's really bad. "All your network are belong
>to us." I don't *think* this particular machine is compromised, but
>how would I know?
You are on Usenet. Most mainstreamers regard Usenet as populated by kooks.
They think listservs and majordomos are safer, more productive, less
porn infested than news groups. It amazes me that mailing lists are
still big deals in some circles {mainframish but also the little old ladies}.
This is the era of tweets, Python, Perl, Java....
--
Looking for an H-912 (container).
(see below)
<eug...@cse.ucsc.edu> wrote:
> I saw NT (2000) on the USS Hopper on its commissioning as well as some
> Aegis cruiser (the Little Rock maybe?). I suspect that certain parts of
> the DoD alread know this is a dumb idea, but they are powerless to
> develop a new OS much less update Ada.
I'm puzzled by this remark. The Ada update process delivered revised
standards in 1995 and 2007; work on the next version is proceeding apace.
--
Bill Findlay
<surname><forename> chez blueyonder.co.uk
Robert Myers
> In article <8ec6cac7-5e12-4773-8c01-5bfa2a122...@v20g2000yqm.googlegroups.com>,
> Robert Myers <rbmyers...@gmail.com> wrote:
>
> >It *can't* be described as Microsoft's fault? Eugene has told us here
> >how important it is not to insult Bill Gates because he can do the
> >industry so much good.
> I didn't say that.
I don't want to dig up the post. You basically said to lay off Gates
because he was the only one people would listen to. Now you're saying
people won't listen to him, either. *That* I believe.
<snip>
> >Really? Why not tell the DoD that having
> >Windows on SSBN's is a *really* dumb idea?
> I saw NT (2000) on the USS Hopper on its commissioning as well as some
> Aegis cruiser (the Little Rock maybe?). I suspect that certain parts of
> the DoD alread know this is a dumb idea, but they are powerless to
> develop a new OS much less update Ada. Not do much to influence
> hardware development. Don't forget all the laptops used by the heads of
> the US Administrator.
I almost worked at JHU/APL. At one point, I thought Aegis was in a
class with the Sgt. York gun. I suppose I was wrong.
> Most fire control systems don't use MS as its not cleared for real-time
> applications. The space station uses Windows (and Ada).
>
> Different parts of the DoD have responsibility for software which come
> in contact with their people in various ways including Obama's fingers
> (not just his secure version of the Blackbery).
>
> The problem is that it is insufficient to merely say something is a dumb
> idea. You have to have a completely formed alternative and a transition
> plan to get there. Think Metric.
I've already said that I think the only metric the world understands
is money. Fine. Money it shall be. I think we're way past the time
for human factors engineering. Recognize the risk, recognize the
costs, and start putting real dollar signs on screwups. Get Wall
Street interested. I'm sure they can find all kinds of ways of
arbitraging risks so that Nick's fears about small companies being
driven out of business wouldn't be realized. I'll bet the American
Trial Lawyers Association could be interested; maybe siphon some of
the predators away from intellectual property law onto something more
socially useful.
I agree. No more happy talk. There's more than enough talent just
posting here to solve the technical problems. The real problems are
economic, managerial, and political.
I keep mentioning the DoD because it is the only place I know of that
routinely pays no attention to the real world constraints that stop
most efforts at reform. The DoD has done better than most with
integrating women and racial minorities into its ranks. If it can do
that, it can surely fix the way software is written.
> >Is that less important than a vaccine for malaria?
>
> Are we going to see detonations?
I don't think that's a likely scenario. I think that social
engineering, compromise of ops details, blackmail, or just scaring the
bejeebers out people who think that SSBN's are invulnerable are more
likely scenarios.
<snip>
> You are on Usenet. Most mainstreamers regard Usenet as populated by kooks.
> They think listservs and majordomos are safer, more productive, less
> porn infested than news groups. It amazes me that mailing lists are
> still big deals in some circles {mainframish but also the little old ladies}.
>
> This is the era of tweets, Python, Perl, Java....
I'm so totally uncool.
Robert.
James Harris
> On Jul 31, 6:26 pm, eug...@cse.ucsc.edu (Eugene Miya) wrote:> In article <2db2edaf-eb11-4ca0-9699-800becd4d...@p29g2000yqh.googlegroups.com>,
> > Robert Myers <rbmyers...@gmail.com> wrote:
...
> > >The spam is a minor irritation for me, not worth any effort to fix.
>
> > You are leaving it to news admins who set up cancel bots.
>
> You bet I am. The idea that I should pay a fee and have to cope with
> another password just to read Usenet is ridiculous.
>
> > >I'll look for the good stuff, anyway.
>
> > >My concern is that the good stuff will disappear.
>
> > You want to sit passively?
>
> That is the first time anyone has ever accused me of being too little
> involved. I do, in fact, report spam to Google, for all the good it
> does.
So do I. Have you noticed that spam reports about some newsgroups seem
to be ignored while reports about other groups are acted on quickly
and effectively?
I can only guess that different people within Google receive these
reports and some make more effort than others.
James
Eugene Miya
If you view Ada as that Standard (1815) which went forward from 1983,
the scope of the language may appear to be nicely advancing. But if you
go back further and further, the DOD higher order language working group,
the APSE got left on the way side (not a bad thing in some eyes),
and the hardware standardization effort was practically still born.
The US Govt in the DOD, the DOE, et al were told NOT to work on
developing OSes, languages, etc. None of these will help the state of
software. This is not to say that the DOD can and will help, it's just
a player (the military-industrial complex). As a result, they have to
be happpy with what ever or how ever Ada evolves. They are now just
along for the ride. It's the light from the lamp post in the prior
analogy.
Eugene Miya
Robert Myers <rbmye...@gmail.com> wrote:
>On Aug 14, 9:02=A0pm, eug...@cse.ucsc.edu (Eugene Miya) wrote:
>> In article <8ec6cac7-5e12-4773-8c01-5bfa2a122...@v20g2000yqm.googlegroups=
>.com>,
>> Robert Myers =A0<rbmyers...@gmail.com> wrote:
>> >It *can't* be described as Microsoft's fault? =A0Eugene has told us here
>> >how important it is not to insult Bill Gates because he can do the
>> >industry so much good.
>
>> I didn't say that.
>
>I don't want to dig up the post. You basically said to lay off Gates
>because he was the only one people would listen to. Now you're saying
>people won't listen to him, either. *That* I believe.
Do you think I as a flamer would tell other people NOT to flame others
when it's the nature of Usenet? Gates and other high profile rich
software types speak for their portion of the industry. Other
industries only care about the money. Most people around here would say
that Gates doesn't speak for them (opinion), but the reality is whether
we like it or not, he's seen as the voice of software (sad, isn't it?).
Yeah, you can cite a list of other guys from Jobs down to Stallman to
perhaps your most obscure local expert (Say Iverson of APL). Any opinion
you and I may have will be measured against Mr Big by the outside world.
Does the rest of the world ignore him? They now for a number of reasons.
For instance most people in the US would rather deal with GM, which got
kicked off the Dow. They have less interest in Cisco which got on the
Dow and requires a greater degree of education than GM. Cisco has to
track what ever MS does. MS a little less wat Cisco does.
>> >Really? =A0Why not tell the DoD that having
>> >Windows on SSBN's is a *really* dumb idea? =A0
>
>> I saw NT (2000) on the USS Hopper on its commissioning as well as some
>> Aegis cruiser (the Little Rock maybe?). =A0I suspect that certain parts o=
>f
>> the DoD alread know this is a dumb idea, but they are powerless to
..
>I almost worked at JHU/APL. At one point, I thought Aegis was in a
>class with the Sgt. York gun. I suppose I was wrong.
Aegis appears quite successful. I don't know how success is fully
measured in this case, but Aegis ships continue to be made, and linked
to planes and other weaponry.
The York prototype is still out in the desert N of Yuma, AZ along a
drive way. They were likely too cheap and had a poor model of how
difficult their problem was. The DOD commonly wastes all kinds of money.
Too long gone to bother harping. At best a teachable lesson.
>> The problem is that it is insufficient to merely say something is a dumb
>> idea. =A0You have to have a completely formed alternative and a transition
>> plan to get there. =A0Think Metric.
>
>I've already said that I think the only metric the world understands
>is money. Fine. Money it shall be. I think we're way past the time
>for human factors engineering. Recognize the risk, recognize the
>costs, and start putting real dollar signs on screwups. Get Wall
>Street interested. I'm sure they can find all kinds of ways of
>arbitraging risks so that Nick's fears about small companies being
>driven out of business wouldn't be realized. I'll bet the American
>Trial Lawyers Association could be interested; maybe siphon some of
>the predators away from intellectual property law onto something more
>socially useful.
I have to try to thread back to what you guys were saying about the
state of software. Part of the problem is hardware.
>I agree. No more happy talk. There's more than enough talent just
>posting here to solve the technical problems. The real problems are
>economic, managerial, and political.
I doubt that we are past the time of human factors science.
I doubt that the talent reading this is necessarily great talent.
It's the talent which has time to read and post to Usenet.
I don't even have that. I have to fix a flat on my bike, but that has
to wait until I get home and have a buck for immersion (of the tube).
>I keep mentioning the DoD because it is the only place I know of that
>routinely pays no attention to the real world constraints that stop
>most efforts at reform. The DoD has done better than most with
>integrating women and racial minorities into its ranks. If it can do
>that, it can surely fix the way software is written.
I am not so certain of the "surely." The DoD likes to pride
"integration" (Powell was a finne general) but it still remains a hot
bed of social problems. what did you say?
>The real problems are economic, managerial, and political.
Sort of.
I am not certain that reform is the issue.
The DOD has its own little world. They have their set of fiefdoms.
>> >Is that less important than a vaccine for malaria?
>> Are we going to see detonations?
>
>I don't think that's a likely scenario. I think that social
>engineering, compromise of ops details, blackmail, or just scaring the
>bejeebers out people who think that SSBN's are invulnerable are more
>likely scenarios.
Well I certainly know people who get paid much more than me who are
surprised it has happened yet. And continuing 9/11 discussion
(non-nuclear), I suspect Capitol Hill would be the next plane target
(they are quite consistent).
Don't knock vaccines. It may get him a Nobel Peace prize.
>> You are on Usenet. Most mainstreamers regard Usenet as populated by kooks.
>> They think listservs and majordomos are safer, more productive, less
>> porn infested than news groups.
>>
>> This is the era of tweets, Python, Perl, Java....
>
>I'm so totally uncool.
Depends how far you want to be left behind and why.
We haven't all taken the blue pill or the red pill.
We have a mix of both. It's Multipicity. It's getting the agents to
interoperate. It's what do with it.
Eugene Miya
James Harris <james.h...@googlemail.com> wrote:
>> report spam to Google,
>
>So do I. Have you noticed that spam reports about some newsgroups seem
>to be ignored while reports about other groups are acted on quickly
>and effectively?
>
>I can only guess that different people within Google receive these
>reports and some make more effort than others.
You can't expect consistency of spam handling across the net at this time.
For the most part, including G, yahoo, hotmail, and other ISPs they will
only deal with people and things in their domain. It's not like the
days where Stoll would track back and find some one. People don't even
have time to complain about open relays. It is amusing when you have
other power. You can tweak from the side. Then you can get job offers.
(see below)
<eug...@cse.ucsc.edu> wrote:
> In article <C6ABC179.1208DC%yald...@blueyonder.co.uk>,
> (see below) <yald...@blueyonder.co.uk> wrote:
>> On 15/08/2009 02:02, in article 4a85fb26$1@darkstar, "Eugene Miya"
>> <eug...@cse.ucsc.edu> wrote:
>>> I saw NT (2000) on the USS Hopper on its commissioning as well as some
>>> Aegis cruiser (the Little Rock maybe?). I suspect that certain parts of
>>> the DoD alread know this is a dumb idea, but they are powerless to
>>> develop a new OS much less update Ada.
>>
>> I'm puzzled by this remark. The Ada update process delivered revised
>> standards in 1995 and 2007; work on the next version is proceeding apace.
>> --
>> Bill Findlay
>
> If you view Ada as that Standard (1815) which went forward from 1983,
> the scope of the language may appear to be nicely advancing.
> The US Govt in the DOD, the DOE, et al were told NOT to work on
> developing OSes, languages, etc. None of these will help the state of
> software. This is not to say that the DOD can and will help, it's just
> a player (the military-industrial complex). As a result, they have to
> be happpy with what ever or how ever Ada evolves. They are now just
Who told the DoD that?
Eugene Miya
>> developing OSes, languages, etc. None of these will help the state of
>> software. This is not to say that the DOD can and will help, it's just
>> a player (the military-industrial complex). As a result, they have to
>> be happpy with what ever or how ever Ada evolves. They are now just
In article <C6AF987D.120E35%yald...@blueyonder.co.uk>,
(see below) <yald...@blueyonder.co.uk> wrote:
>Who told the DoD that?
That was before my time, but I suspect it goes back to Rep. Brooks from
one of the New England States. But the signal also comes numerous
sources not only in the Legistative Branch but also the Executive Branch
like the OMB. The DOD is a bigger fish than I have any interest in
frying. I have my own agency to deal with, and I also see the DOE close by.
It's the problem inherent with "Mission" oriented agencies.
This kind of work is supposed to be left to industry.
Never mind the circular logic to that.
If you are going to solve "the software problem,"
you have to solve it as a sort of complete system.
Govt isn't supposed to compete with industry.... What ever that means.
Let me try to catch up with one more of Robert's posts. But I have to
head up to SF.
(see below)
<eug...@cse.ucsc.edu> wrote:
>>> The US Govt in the DOD, the DOE, et al were told NOT to work on
>>> developing OSes, languages, etc. None of these will help the state of
>>> software. This is not to say that the DOD can and will help, it's just
>>> a player (the military-industrial complex). As a result, they have to
>>> be happpy with what ever or how ever Ada evolves. They are now just
>
> In article <C6AF987D.120E35%yald...@blueyonder.co.uk>,
> (see below) <yald...@blueyonder.co.uk> wrote:
>> Who told the DoD that?
>
> That was before my time, but I suspect it goes back to Rep. Brooks from
> one of the New England States. But the signal also comes numerous
> sources not only in the Legistative Branch but also the Executive Branch
> like the OMB. The DOD is a bigger fish than I have any interest in
> frying. I have my own agency to deal with, and I also see the DOE close by.
> It's the problem inherent with "Mission" oriented agencies.
>
> This kind of work is supposed to be left to industry.
> Never mind the circular logic to that.
Indeed.
> If you are going to solve "the software problem,"
> you have to solve it as a sort of complete system.
> Govt isn't supposed to compete with industry.... What ever that means.
But surely it commissions industry to do its bidding?
It certainly does for the development of weapons systems.
I suspect that vested interests were at work.
No, really! I do!
Robert Myers
> On 18/08/2009 01:57, in article 4a89ee57$1@darkstar, "Eugene Miya"
>
>
>
> <eug...@cse.ucsc.edu> wrote:
> >>> The US Govt in the DOD, the DOE, et al were told NOT to work on
> >>> developing OSes, languages, etc. None of these will help the state of
> >>> software. This is not to say that the DOD can and will help, it's just
> >>> a player (the military-industrial complex). As a result, they have to
> >>> be happpy with what ever or how ever Ada evolves. They are now just
>
> > (see below) <yaldni...@blueyonder.co.uk> wrote:
> >> Who told the DoD that?
>
> > That was before my time, but I suspect it goes back to Rep. Brooks from
> > one of the New England States. But the signal also comes numerous
> > sources not only in the Legistative Branch but also the Executive Branch
> > like the OMB. The DOD is a bigger fish than I have any interest in
> > frying. I have my own agency to deal with, and I also see the DOE close by.
> > It's the problem inherent with "Mission" oriented agencies.
>
> > This kind of work is supposed to be left to industry.
> > Never mind the circular logic to that.
>
> Indeed.
>
> > If you are going to solve "the software problem,"
> > you have to solve it as a sort of complete system.
> > Govt isn't supposed to compete with industry.... What ever that means.
>
> But surely it commissions industry to do its bidding?
> It certainly does for the development of weapons systems.
>
> I suspect that vested interests were at work.
> No, really! I do!
Only if you believe that Reagan-era loathing for government initiative
and uncritical embrace of private enterprise is a vested interest.
The usual suspect to round up is William Perry. The key step in the
history can be found (for example) in this document
http://www.wise-intern.org/journal/1998/SHERTZER.PDF
Robert.
Eugene Miya
>>>> developing OSes, languages, etc. None of these will help the state of
>>>> software. This is not to say that the DOD can and will help, it's just
>>>> a player (the military-industrial complex). As a result, they have to
>>>> be happpy with what ever or how ever Ada evolves. They are now just
>>
>>
>> It's the problem inherent with "Mission" oriented agencies.
>>
>> This kind of work is supposed to be left to industry.
>> Never mind the circular logic to that.
In article <C6AFB5EF.120E6D%yald...@blueyonder.co.uk>,
(see below) <yald...@blueyonder.co.uk> wrote:
>Indeed.
>
>> If you are going to solve "the software problem,"
>> you have to solve it as a sort of complete system.
>> Govt isn't supposed to compete with industry.... What ever that means.
>
>But surely it commissions industry to do its bidding?
It issues contracts. That is part of the problem.
>It certainly does for the development of weapons systems.
Have you been following the revolving door discussions in Congress?
>I suspect that vested interests were at work.
>No, really! I do!
They mostly are, but there are other forces like technology at work.
Ada is a cost saving measure. It does help in a way, but it also
freezes out newer tools developed like the Java Virtual Machine (VMware
can sort of make up for some of this), ideas in various new languages,
etc. I did see the paper in the late 1990s asking the DOD to ease up on
Ada, on the use of shielded cabling to use Ethernet in F-16s instead,
and all COTS --Commercial Off The Shelf stuff as much as possible.
Do I think that Boeing, N-G, GE et al don't want to change?
Sure, I hear the mechanic in the current GE ad on TV ask for an English
units wrench.
And do you think that they want to warp their suppliers for the things
they want as opposed to what other consumers might want?
This started as a thread about DARPA in part, and notice there was
no Ada requirement on the Grand Challenge. That's DARPA.
Can you imagine the response had there been?
Robert Myers
> This started as a thread about DARPA in part, and notice there was
> no Ada requirement on the Grand Challenge. That's DARPA.
> Can you imagine the response had there been?
I don't think an Ada requirement would have been a good idea for
DARPA. The correct place to pursue Ada is DoD, which is why I brought
a three-letter agency into the discussion.
Robert.
(see below)
<eug...@cse.ucsc.edu> wrote:
Ada is a life saving measure, so far as I am concerned.
> freezes out newer tools developed like the Java Virtual Machine (Vmware
There are Ada compilers that generate JVM and .NET code. No freezing there.
> can sort of make up for some of this), ideas in various new languages,
Ada in some ways is ahead of "new" languages.
> etc. I did see the paper in the late 1990s asking the DOD to ease up on
> Ada, on the use of shielded cabling to use Ethernet in F-16s instead,
> and all COTS --Commercial Off The Shelf stuff as much as possible.
>
> Do I think that Boeing, N-G, GE et al don't want to change?
> Sure, I hear the mechanic in the current GE ad on TV ask for an English
> units wrench.
Sometimes you are so gnomic, Eugene, I have no idea what you mean. 8-)
> And do you think that they want to warp their suppliers for the things
> they want as opposed to what other consumers might want?
They do that all the time, surely?
Eugene Miya
Robert Myers <rbmye...@gmail.com> wrote:
>On Aug 18, 5:47=A0pm, eug...@cse.ucsc.edu (Eugene Miya) wrote:
>> This started as a thread about DARPA in part, and notice there was
>> Can you imagine the response had there been?
>
>I don't think an Ada requirement would have been a good idea for
>DARPA. The correct place to pursue Ada is DoD, which is why I brought
>a three-letter agency into the discussion.
DARPA is in the DOD.
That's the irony.
The DOD is smart enough not to be dogmatic.
Unless you are a long time contractor.
I am trying to swim back to some of your other posts....
Eugene Miya
(see below) <yald...@blueyonder.co.uk> wrote:
>>> I suspect that vested interests were at work.
>>> No, really! I do!
>>
>> They mostly are, but there are other forces like technology at work.
>> Ada is a cost saving measure. It does help in a way, but it also
>
>Ada is a life saving measure, so far as I am concerned.
As a generality.
This is why standards are important.
>> freezes out newer tools developed like the Java Virtual Machine (Vmware
>
>There are Ada compilers that generate JVM and .NET code. No freezing there.
Now there are.
>> can sort of make up for some of this), ideas in various new languages,
>
>Ada in some ways is ahead of "new" languages.
A lot of thought went into it and its competitors at the time. I left
the software engineering/language arena some time ago. Some historical
revisionism appears to have taken place.
It's not necessarily that newer language are superior. A feature or two
might be interesting. A lot can be said of both. The problem for the
DOD is Who will be creating their next generation of Ada programmers?
e.g, WVU? Has the money at CMU's SEI been well spent? I think people
could claim mixed returns. Software is still cottage craft. I have to
get back to the State thread.
>> etc. I did see the paper in the late 1990s asking the DOD to ease up on
>> Ada, on the use of shielded cabling to use Ethernet in F-16s instead,
>> and all COTS --Commercial Off The Shelf stuff as much as possible.
>>
>> Do I think that Boeing, N-G, GE et al don't want to change?
>> Sure, I hear the mechanic in the current GE ad on TV ask for an English
>> units wrench.
>
>Sometimes you are so gnomic, Eugene, I have no idea what you mean. 8-)
See if you can YouTube the current set of General Electric commercials
and find the one where a jet engine mechanic asks for an 11/16 inch
wrench. A GE wind turbine man tosses it between say half a dozen
different GE people/applications to the original guy. This all sounds
innocent enough to older folk. You don't have to field questions by my
friends why the Constellation Program stays with English units. I was
just talking with a friend here at work who works on heat shields and
she uses Metric. Programming language (the DOD/Ada thread) has
statically checked units 2-3 decades ago but that feature was not adopted.
The Quintin Tarantino generation gets it (sometimes).
>> And do you think that they want to warp their suppliers for the things
>> they want as opposed to what other consumers might want?
>
>They do that all the time, surely?
My govt managers are trying to influence Intel in ways Intel has no
interest, and they can't fathom why we (the govt) have no influence
(radiation hardening of the Intel line).
No political appeal (only merely making a buck) in any of this.
(see below)
<eug...@cse.ucsc.edu> wrote:
> It's not necessarily that newer language are superior. A feature or two
> might be interesting. A lot can be said of both. The problem for the
> DOD is Who will be creating their next generation of Ada programmers?
> e.g, WVU? Has the money at CMU's SEI been well spent? I think people
> could claim mixed returns. Software is still cottage craft. I have to
> get back to the State thread.
Yes, the stagnation of the s/w industry is quite shameful, really.
> See if you can YouTube the current set of General Electric commercials
> and find the one where a jet engine mechanic asks for an 11/16 inch
> wrench. A GE wind turbine man tosses it between say half a dozen
> different GE people/applications to the original guy. This all sounds
> innocent enough to older folk. You don't have to field questions by my
> friends why the Constellation Program stays with English units. I was
> just talking with a friend here at work who works on heat shields and
> she uses Metric. Programming language (the DOD/Ada thread) has
> statically checked units 2-3 decades ago but that feature was not adopted.
> The Quintin Tarantino generation gets it (sometimes).
Ah, I see.
Do you think the culture of conservatism in the US affects the way people
think about non socio-political matters? On my visits I've often noticed how
old-fashioned a lot of things seemed (fire appliances, telephones, women's
hair styles, cars, ....).
>>> And do you think that they want to warp their suppliers for the things
>>> they want as opposed to what other consumers might want?
>>
>> They do that all the time, surely?
>
> My govt managers are trying to influence Intel in ways Intel has no
> interest, and they can't fathom why we (the govt) have no influence
> (radiation hardening of the Intel line).
> No political appeal (only merely making a buck) in any of this.
Don't the DoD have enough bucks to get any bang they want?
Eugene Miya
(see below) <yald...@blueyonder.co.uk> wrote:
>Bill Findlay
>On 19/08/2009 17:37, in article 4a8c1c54$1@darkstar, "Eugene Miya"
><eug...@cse.ucsc.edu> wrote:
>> It's not necessarily that newer language are superior. A feature or two
>> might be interesting. A lot can be said of both. The problem for the
>> DOD is Who will be creating their next generation of Ada programmers?
>> e.g, WVU? Has the money at CMU's SEI been well spent? I think people
>> could claim mixed returns. Software is still cottage craft. I have to
>> get back to the State thread.
>
>Yes, the stagnation of the s/w industry is quite shameful, really.
It's a human problem. Some of the factors are clearly cultural.
Andy Tanenbaum last railed about it in comp.os.research.
I saw some of it in vectorizing compilers in the 80s (CDC and Cray sort
of rested on their laurels until the Japanese did their homework, then
Convex and Alliant had to come in and IBM bought their technology from
others). Water under the bridge boring stuff now.
The US car industry is the same way.
>> See if you can YouTube the current set of General Electric commercials
>> and find the one where a jet engine mechanic asks for an 11/16 inch
>> wrench.
>> Quintin Tarantino: "Royal with cheese..."
>
>Ah, I see.
>Do you think the culture of conservatism in the US affects the way people
>think about non socio-political matters? On my visits I've often noticed how
>old-fashioned a lot of things seemed (fire appliances, telephones, women's
>hair styles, cars, ....).
You know, I heard a Swiss physicist say the exact same thing walking up
Columbus St in SF from the Consulate to their house where we ate crepes.
We passed chocolate shops and schools and libraries, and they commented
the schools were like they were in the 60s, sort of like Zurich airport
(Flughafen). Some of the US complacency came from winning WWII and the
Cold War. Don't forgot that for decades it was "the market." It's a
new century now and the question is how the US will change.
It's all handling complexity, Bill.
>>>> And do you think that they want to warp their suppliers for the things
>>>> they want as opposed to what other consumers might want?
>>>
>>> They do that all the time, surely?
>>
>> My govt managers are trying to influence Intel in ways Intel has no
>> interest, and they can't fathom why we (the govt) have no influence
>> (radiation hardening of the Intel line).
>> No political appeal (only merely making a buck) in any of this.
>
>Don't the DoD have enough bucks to get any bang they want?
A friend tried to construct the most expensive computer ever for Ft Meade.
Even they have their limits. I wonder if GCHQ, MI-6 or MI-5 could have
paid for that proof of concept. They came to us to see if we have money
to contribute which was a little disappointment to them. Very
interesting stuff in some ways but architecturally conservative (very
device advanced).
The problem with advanced stuff is paying for it. No export market.
After F-14 (Iran), F-15 (Saudi), AWACS debacles (Saudi), the USAF,
to say nothing of the other uniformed services, wants to go up against
older US hardware. The C-5, C-17, C-141 didn't go the way of
the C-135 and the Boeing 707. And the 747 did fine w/o the DOD even if
tricky.
My managers and I had an amusing discussion about arrogance today.
I have to run. An aero get together, and I am still trying to fathom
what Robert is trying to say.
Robert Myers
>
> I have to run. An aero get together, and I am still trying to fathom
> what Robert is trying to say.
>
I've talked about only one thing that's really deep: do you see new
physics when you do larger calculations?
I know of exactly one example where it is clear that you do. I'm sure
there are other examples, but this is the example I think I
understand.
The physics of turbulence are present at even low Reynolds numbers.
The problem is, most of the interesting stuff becomes clear only when
you get to meaningful separation of scales. No matter what you do,
you can't do actual fluid mechanics at a Reynolds number much higher
than the grid Reynolds number--the number of grid points in a single
dimension. When Steve Orszag did calculations a long time ago, he got
the result that we'd never really get to what I now interpret as a
meaningful separation of scales by direct calculation. I've had the
pleasure of being dissected by him in person, so I'm not (or wasn't) a
rank amateur on this subject.
Just to make sure that I'm not making a complete fool of myself
because I've been out of the loop so long, I consulted Wikipedia (an
unimpeachable source of common wisdom, if not of absolute truth):
http://en.wikipedia.org/wiki/Direct_numerical_simulation
It all looks very familiar.
The arbitrary things you do to make calculations even *possible* (no
floating overflows, for example) at meaningful Reynolds numbers almost
always determine the pseudo-physics you are "simulating." That is to
say, most computational fluid mechanics reflects the biases and ad hoc
assumptions of the modeler, and not necessarily anything to do with
the physics and/or mathematics of the Navier-Stokes equations. Every
scheme that localizes the approximation of differential operators has
a profound effect on the simulated physics. If you get it right, it's
only by accident.
One gets around these problems by "renormalization." CFD guys call it
turbulence modeling, but the same problem has to arise in any
seriously nonlinear physics where there is a large range of scales.
The big computers we are now building will not ever make any
meaningful progress on this fundamental issue. We might as well stop
with the machine I'm working on right now, or maybe just a tad bigger.
If you're *not* doing that problem, then I am puzzled as to what you
*are* doing with your big machines. I guess that seismic modeling can
benefit from lots of detail even with local differencing, although I'm
not sure if that's reality or just hope. In every "interesting"
problem that I know, the interesting physics are global, not local,
but they inevitably depend in detail on local behavior. If you're
getting away with small global bandwidth, you're either doing
uninteresting physics or telling elaborate fibs to yourself and anyone
who pays for what you're doing.
That's at the core of my contempt for what's happening in (say) global
climate prediction and for the building of computers that almost force
localized approximations to differential operators. In the case of
the Atlantic thermohaline problem, the physics were captured long
before there was a Cray-I and I really don't know what progress the
big calculations have made.
Oil companies, structural engineers, and even aerodynamicists who need
lots of flops are getting what they need from racks of COTS hardware
with COTS fabric. If you're not going to move beyond what they do,
why bother with DARPA, NSF, the weapons labs, or anyone else? I'm
sorry the strategic stockpile problem is classified. I'd dearly love
to understand what physics they think they're modeling.
There are lots of details about what is and what isn't known and what
you can try that's already been tried, and so on. I don't want to try
to turn this into a physics/mathematics forum. I haven't ever had a
chance to look in detail at the successes and failures of (say)
protein folding simulations, but I'll be very surprised if the same
problem isn't lurking there.
Robert.
Terje Mathisen
> If you're *not* doing that problem, then I am puzzled as to what you
> *are* doing with your big machines. I guess that seismic modeling can
> benefit from lots of detail even with local differencing, although I'm
> not sure if that's reality or just hope. In every "interesting"
Seismic can (and do!) use a couple of orders of magnitude improvement
and gain real benefits:
The first order is going from "post-stack" to "pre-stack" processing,
this is what makes it possible to get visual detail of formations
beneath a salt dome (or other structure with very different sound
propagation numbers). The pre- term refers to using the seismic lines
directly, vs first stacking them together and thereby reducing the
amount of data by an order of magnitude.
The other new wrinkle I've read about is to do something that sounds
like direct ray tracing, calculating directly the behavior of both the
downward pulse and the reflected waves.
> Oil companies, structural engineers, and even aerodynamicists who need
> lots of flops are getting what they need from racks of COTS hardware
Not quite: At least in the codes employed by Statoil & Hydro you split
the problem into two phases, balancing the cutover point so they take
approximately the same amount of wall clock time.
The first stage needs a _very_ big global address space with as close to
UMA memory access as you can get, it does all the partitioning work so
that the second stage can indeed run on huge racks of COTS gear.
Terje
--
- <Terje.Mathisen at tmsw.no>
"almost all programming can be viewed as an exercise in caching"
Robert Myers
wrote:
>
> The first stage needs a _very_ big global address space with as close to
> UMA memory access as you can get, it does all the partitioning work so
> that the second stage can indeed run on huge racks of COTS gear.
>
If UMA means that all the processors can be fed by data at roughly the
same speed even when bandwidth demands are high, and not maybe more
than a factor of five higher than local memory access, then all my
complaints go away. That condition is not met by the big machines
that have gotten the most press.
Robert.
Eugene Miya
Robert Myers <rbmye...@gmail.com> wrote:
>On Aug 20, 8:41=A0pm, eug...@cse.ucsc.edu (Eugene Miya) wrote:
>> I have to run. An aero get together, and I am still trying to fathom
>> what Robert is trying to say.
>
>I've talked about only one thing that's really deep: do you see new
>physics when you do larger calculations?
That's a good question.
Actually you are saying more than that. 1-2 of your posts have revealed
that you are coming from a community unlike most netters.
Short answer is yes. But it's not my science. And I can think of refs
to the late 80s and 90s on that. But I'll get to them tomorrow as I'm
in a break with 2 more sets of meetings to attend. Just a day of meetings.
Eugene Miya
Robert Myers <rbmye...@gmail.com> wrote:
>> >Time to get off your hands, lose your pride, and say something.
On Jul 31, 6:45=A0pm, eug...@cse.ucsc.edu (Eugene Miya) wrote:
>> Have you signed Non-disclosure?
>
>Do you know about I.F. Stone and his weekly? Of course you do. Not
Nope not really.
It was a rhetorical question.
>only was he black-listed, he was also nearly deaf and so couldn't
>participate in the cocktail circuit. Most doors were closed to him,
>anyway.
>
>Even so, he was the first journalist to expose the deliberate
>fabrications that resulted in the Gulf of Tonkin resolution and the
>disastrous expansion of the Viet Nam war. Entirely from publicly-
>available documents.
The Maddox and Turner Joy.
The details just before my time.
>No, I haven't signed any non-disclosure agreements recently, nor have
>I had a security clearance for a long time. I don't need any of those
>to calculate a bi-section bandwidth, nor to look at the performance of
>our "super" computers on some kinds of calculations, nor even to find
>critical admissions from the national laboratories as to the inherent
>limitations of the kinds of computers that make up almost the entire
>"Top" 500 list.
>The state of software is a national disgrace. It isn't always
International as Nick points out.
>possible to separate the software issues from the hardware issues, and
>I have really very little interest in wading through endless and
>pointless discussions about programming style and languages. The
It's largely, because people here are powerless.
>*only* place I know of where the issue has been addressed in any
>reasonable fashion is avionics, and it's an issue that's not really on
>the radar of comp.arch.embedded.
That's very telling.
Did you attend Ohio Univ.?
Very few avionics people here.
Comp.arch is dominated by academics and telecommunications types
(from back when AT&T employed over 3M people). It's a Unix artifact.
This is why you don't see much avionics. Comparatively few aerospace
people here. It's a cultural thing. It's the difference between the
attempt at a computer industry in Southern California vs. the computer
industry in Northern CA (Steven Levy wrote a little about it).
Similarly when you look at the comp.sys.* hierarchy, it's a lopsided
collect of computer companies not representative of corporations at that
time. It's also why comp.dsp exists. Groups exist for what people
wanted threads to read.
In some ways sci.aeronautics would be better for avionics. It was just
too much effort to moderate it.
>You take some of this personally. I wish you wouldn't.
It's just business.
Such limited fun here now these days.
Eugene Miya
Robert Myers <rbmye...@gmail.com> wrote:
>On Jul 31, 9:41=A0pm, eug...@cse.ucsc.edu (Eugene Miya) wrote:
>> people doing heady things here is like listening to Barb in a.f.c. and
>> taking everything she says as word of God.
>
>No need to worry. I can't stand Barb, and she can't stand me. It's
>why I don't go near a.f.c.
;^)
>As to the rest of it (egos, posturing, name-dropping), I wish you
>wouldn't waste your time.
Oh, you're no fun.
Not much a waste of time.
Eugene Miya
In article <95c36a4a-fd7f-41ba...@k19g2000yqn.googlegroups.com>,
Robert Myers <rbmye...@gmail.com> wrote:
>On Aug 1, 3:58=A0am, n...@cam.ac.uk wrote:
>> In article <662a19d7-f6e4-4562-8143-af9cdc8fa...@w6g2000yqw.googlegroups.=
>com>,
>> Robert Myers =A0<rbmyers...@gmail.com> wrote:
>> >The state of software is a national disgrace. =A0It isn't always
>> >possible to separate the software issues from the hardware issues, and
>> >I have really very little interest in wading through endless and
>> >pointless discussions about programming style and languages. =A0The
>> >*only* place I know of where the issue has been addressed in any
>> >reasonable fashion is avionics, and it's an issue that's not really on
>> >the radar of comp.arch.embedded.
>>
>> No, it's not. =A0It's an international disgrace!
>>
>I'm sure you're right, but I don't know where to point a finger in the
>international arena. I suspect that not even the entire international
>community can squander the level of financial resources that the U.S.
>does and still make no apparent (or even negative) progress.
I doubt we need to point almost anywhere but the USA. This was
something Andy Tanenbaum railed about in comp.os.research. Just go back
in the archives (dismal, Andy just gave up and left, he was reading too
much into c.o.r.). The US, like cars, perceives itself as "the market."
>Eugene
>has pointed to one aspect of the real problem, which is that the
>decision-makers with money to spend (or budget) are more concerned (or
>perhaps exclusively concerned) with protecting their fiefdoms than
>with pursuing any more laudable goal.
It's not solely money, but there are fiefdoms.
Who lauds anything in the US? We self pat ourselves on the back in this
country. Take a look at selfpromotion.com (not that I endorse that).
People (some) want to make a buck off some of their software.
Yet, it's not enough to even give it away free in many cases.
Nick wrote:
>> And the really dispiriting thing about those endless and pointless
>> debates is that the most plausible solution does lie down that
>> route - just not in that direction.
>>
>Some of the smartest theoreticians in this area are on the payroll of
>the world's largest software company. Now *that's* discouraging.
Well, when you were starting this, I was in Redmond, WA. Tony Hoare has
to eat. It's a sprawling campus now.
Consider their theory as inadequate.
I am not certain that I would have said what Nick said, the way he said it.
That "route" IS a direction.
Off to 2 more meetings.....
Mayan Moudgill
> The state of software is a national disgrace.
I don't agree. I actually think software is in pretty decent state.
What yardstick are you using? What other product design activity would
you compare it against?
Robert Myers
When something goes wrong in the design of a structure, and it fails
catastrophically, there are consequences.
One looks to see the signature and seal of what *licensed* structural
engineer is on the document. One examines the design for conformity
to codes. One looks at the inspection history and who signed what.
Lawyers hire consulting engineers and hang on their every words, as
there is money, sometimes big money, at stake.
I have a friend who has designed a clever and commercially-successful
product. Good science and engineering have gone into that product,
which has competitors. There is no code because it's a niche
product. He wraps himself in liability insurance. If his product
fails and someone is injured or worse as a consequence, he will have
to defend himself, and the mere fact that he has been clever beyond
the capabilities of most mortals will do him no good.
Commercial and military aircraft involve a degree of design
sophistication that is, in itself, breathtaking. What is even more
amazing is the large numbers of different disciplines that must work
together successfully to produce a product that works and can be sold
into a market with razor thin (or, in some cases, negative) margins.
Boeing tried to shop out a lot of work on the Dreamliner and
discovered what should have been no surprise to anyone: offshoring
critical parts across town is as hard as offshoring to Taiwan, Mexico,
or India. It's a measure of how hard it is to get it right and that,
for all the criticism they endure, they do know how to get it right.
Compare any of this to the state of software. Licensing? That's a
good one. Liability? Bring your reading glasses when buying
software, because the number of rights you give up when you buy
anything is the only thing growing faster than new security exploits.
Management and product integration? Look at the schedule slippage and
design of the user interface for Vista.
If you're a "luser" and not a part of the scam, you get used to things
never quite working the way they're supposed to, debugging practically
every new product you use, and now an endless procession of updates to
fix security holes, the plethora of which updates is a constant
distraction and annoyance. It would be as if bridges were routinely
closed for repairs at random times because they have been discovered
to be unsafe, except that you don't have to use google and pore
through dozens of posts in obscure forums to find out what's really
happening.
I don't want to talk about NASA here, out of deference to Eugene.
Engineers make mistakes, sometimes big ones, in the design of
commercial airplanes, like running high-pressure hydraulic lines right
behind the leading edge of the DC-10. Such mistakes do not go
unnoticed, nor is public discussion of them confined to slashdot.
You can't put a car on the highway in most states without it meeting
certain requirements for road-worthiness. You can't fly airplanes
that can't be certified as airworthy. You can put *anything* on the
internet, and it can become a host to all kinds of misery for people
who had nothing to do with any of it except also to be using the
internet. When a car or an airplane is discovered to be hazardous to
occupants or others, it is recalled and taken out of service.
Software with known exploitable vulnerabilities stays in service for
years and there are no consequences for anyone except those who happen
to be hurt, even if only by hundreds of thousands of pieces of spam
from a bot network or suddenly not being able to use a needed online
service because of a DNS attack. Stock exchanges get shut down. I
don't even want to think about the vulnerabilities of the DNS system
or how much havoc is possible.
I could go on. That's software "engineering." Even more grotesque is
the idea of software as science. It's a science of fads and fashions,
more like haute couture than anything that would pass as serious even
in fields so reckless as cosmology and particle physics. What is
there about the science of software that is as solid and universally
useful as the Fast Fourier Transform or as reliable a way of beginning
as drawing a freebody diagram? It's like twentieth century art, and
nowhere has the prediction of Andy Warhol been more true than that
everybody gets fifteen minutes of fame. String theory has been
furiously promoted and furiously panned. When have the churners-out
of new theories of software "science" ever been so exposed? Where,
for that matter, is the string theory of software "science?"
Pharmaceuticals and the practice of medicine have comparable problems
and a comparably arrogant and self-sustaining and self-congratulatory
culture. That liability lawyers swarm all over drugs and medicine is
solid evidence that liability is itself no panacea.
Ok. That was a rant. I didn't know how to answer without a rant.
Robert.
Mayan Moudgill
> On Aug 26, 9:49 pm, Mayan Moudgill <ma...@bestweb.net> wrote:
>
>>Robert Myers wrote:
>>
>>>The state of software is a national disgrace.
>>
>>I don't agree. I actually think software is in pretty decent state.
>>
>>What yardstick are you using? What other product design activity would
>>you compare it against?
>
>
> When something goes wrong in the design of a structure, and it fails
> catastrophically, there are consequences.
>
Certainly, there are (or should be) consequences proportionate to the
result of the design failure. Unfortunately, there rarely (or maybe we
don't hear about them much).
Consider some recent design flaws in your favorite golden child, aviation:
- 787 wingbox
- airbus 330-200 pitot tubes
Looking at wikipedia category
http://en.wikipedia.org/wiki/Category:Airliner_accidents_and_incidents_caused_by_design_or_manufacturing_errors
I find 16 entries, and they don't include Swissair 111 (inflight
entertainment system upgrade => MPET insulation fire) & TWA 800 (center
wing fuel tank).
Know if anyone got fired for those?
And those are only the ones that have castrophic consequences. Consider
something minor like Alitalia 610 - its windshield cracked. This was
arguably a design flaw - at least 3 777s had the same problem in the
first year, and Boeing did design mods to fix the problem (I believe).
> One looks to see the signature and seal of what *licensed* structural
> engineer is on the document. One examines the design for conformity
> to codes. One looks at the inspection history and who signed what.
> Lawyers hire consulting engineers and hang on their every words, as
> there is money, sometimes big money, at stake.
Correct; I'm guessing TWA 800 cost TWA+Boeing about 0.5 billion dollars.
Whats that - two Boeing 747s?
> I have a friend who has designed a clever and commercially-successful
> product. Good science and engineering have gone into that product,
> which has competitors. There is no code because it's a niche
> product. He wraps himself in liability insurance. If his product
> fails and someone is injured or worse as a consequence, he will have
> to defend himself, and the mere fact that he has been clever beyond
> the capabilities of most mortals will do him no good.
Sure. But software of and by itself can never hurt someone physically. A
device that includes software as a major component can do that. I'm
trying to remember software that has lead to deaths. Therac-25; possibly
AirFrance 447 - anything else? Pretty good record, I'd say.
> Commercial and military aircraft involve a degree of design
> sophistication that is, in itself, breathtaking. What is even more
> amazing is the large numbers of different disciplines that must work
> together successfully to produce a product that works and can be sold
> into a market with razor thin (or, in some cases, negative) margins.
> Boeing tried to shop out a lot of work on the Dreamliner and
> discovered what should have been no surprise to anyone: offshoring
> critical parts across town is as hard as offshoring to Taiwan, Mexico,
> or India. It's a measure of how hard it is to get it right and that,
> for all the criticism they endure, they do know how to get it right.
Really? Looking at the aviation industry in general, I find it has a
pretty poor record of designs. They've been designing planes for ~100
years now [compared to ~50 years for software], the design-space is
small compared to software, they use longer design cycles and larger
teams, and they still don't get it right.
Its also arguable that planes are less complex than high-complexity
software (e.g. OSes). This was definitely so until the fly-by-wire and
dynamically-unstable aircraft - compare the number of parts in a DC-10
to the number of lines of code in OS/360. And then consider the number
of design flaws in the DC-10.
> Compare any of this to the state of software. Licensing? That's a
> good one. Liability? Bring your reading glasses when buying
> software, because the number of rights you give up when you buy
> anything is the only thing growing faster than new security exploits.
> Management and product integration? Look at the schedule slippage and
> design of the user interface for Vista.
As opposed to the latest efforts from Airbus (380 2+ years?), Boeing
(787 2years+?), Lockheed-Martin (F-22 N-years)?
BTW: The F-22 design RFP was issued in 1986 (and that was preceded by
the ATF requirement spec in 1981, so I'm sure design work started
earlier). It took them till 1990/91 to deliver the demo-prototypes. The
first production version was delivered in 2003. So:
- 1981 for initial spec
- 1986 for formal RFP
- 1990 for demo
- 2003 for production
You really think that aviation does a better job than software?
In case you think that this is peculiar to military/DoD, consider the
Airbus A380.
- 1988: prelim design work
- 1990: formal project announce
- 1994: A3XX design started
- 2000: A380 started
- 2005: prototype
- 2007: first delivery
Not much better than the F-22, is it?
> If you're a "luser" and not a part of the scam, you get used to things
> never quite working the way they're supposed to, debugging practically
> every new product you use, and now an endless procession of updates to
> fix security holes, the plethora of which updates is a constant
> distraction and annoyance.
Seen the list of FAA advisories/airworthiness directives. A quick visit
to FAA site reveals 71 directives in the past 60 days. Do you really
think that this reveals rock-solid engineering?
Try comparing designs of like complexity: in general, bridges and cars
(two other examples you've used) are really simple compared to complex
software such as, oh, operating systems.
As an exercise, figure out how many parts there are in a car. Then, find
a program with roughly the same complexity (maybe same number of LOC?).
Compute the man-years spent developing both. Count the number of recalls
for the car and the number of design flaws fixed. Compare that against
the number of bugs in the equivalent program. You might be a little
surprised.
Bernd Paysan
In the USA? I'd have troubles. Housing? Ikea-style "cardboard" houses
with slits under the window so big that you don't need doors¹ ;-). Cars?
With technology as developed in the 60s, just larger? Financial "products"
which rival Tolkien, Pratchett, and Rowling in fantasy?
Compared to that, software is in a pretty decent state. Being finally in an
US managed company now (after a merger), I can see why: Way too impatient.
Good products take time. If you take the time away, you get nothing. Not
just bad products, no products at all. Business classes often ask why
software is always late and over budget? Answer: Because the schedule is
always too aggressive and the budget grossly understated. Unfortunately,
this answer is only written in Dilbert, not in any MBA program.
Don't get me wrong: European managers get the schedule and the budget just
as badly wrong as US managers - there is all way too much wishful thinking
involved. But my experience is that they listen more to their engineers and
finally bite the bullet, and let the project slip and the budget increase.
Of course, European managers also pull the plug when a project takes too
long to achieve the desired quality.
¹) People often say "when houses where build like software, you'd get..."
Well, you'd get houses like they are. With patched roofs and windows that
open only when you kick them. With open (back)doors for all kind of vermin.
--
Bernd Paysan
"If you want it done right, you have to do it yourself"
http://www.jwdt.com/~paysan/
Bernd Paysan
> Its also arguable that planes are less complex than high-complexity
> software (e.g. OSes). This was definitely so until the fly-by-wire and
> dynamically-unstable aircraft - compare the number of parts in a DC-10
> to the number of lines of code in OS/360. And then consider the number
> of design flaws in the DC-10.
Don't compare number of parts to number of lines. An aircraft has many
repetitive parts, and repetitive parts in software are done (or should be
done ;-) by loops and factoring. A jet may have four turbines, each
consists of 16 fans, with 48 blades, and each blade attached with four
rivets (just numbers out of the air). The complexity is not 4*16*48*4 for
the rivets. The fact that the plane has 12288 rivets of that sort has to be
taken into account when you calculate the reliability - how likely is that
four of them fail in a row (or three, if it requires two of them to be ok to
keep the blade in position).
If you want to compare, take the "source" of the aircraft (i.e. the CAD
drawings), and note them down in a way that's similar to software - loops
for repetitive constructs, factoring for common parts, and templates for
parts that are used several times slightly different (e.g. scaled). You'll
probably find that the complexity is much less than you think - and maybe
it's also much less than the aircraft engineers think, because they are not
that much used to factoring as you.
A friend of mine works at Airbus. Last time I saw him, he was testing the
heating system - apparently, the systems are not autonomous, but have a
sensor and a filament attached to a central controller. He said that due to
the fact that each element takes minutes to heat up considerably, the
testing time is "enormous". I told him how we chip designers test chips,
and that the testing of heating filaments and attached sensors is the same
problem (or rather: a very simple sub-problem, since there's no logic
included): You apply a number of orthogonal bit vectors to them, and check
if each of them passes. For ~100 heating elements, 8 test runs are
completely sufficient. I'm not sure if he understood me. My suggestion to
think about arranging the ~100 heating elements conceptually on a 7D
hypercube was apparently also too demanding ;-).
Kai Harrekilde-Petersen
> Mayan Moudgill wrote:
>
>> Robert Myers wrote:
>>
>>> The state of software is a national disgrace.
>>
>> I don't agree. I actually think software is in pretty decent state.
>>
>> What yardstick are you using? What other product design activity would
>> you compare it against?
>
> In the USA? I'd have troubles. Housing? Ikea-style "cardboard" houses
> with slits under the window so big that you don't need doors� ;-). Cars?
> With technology as developed in the 60s, just larger? Financial "products"
> which rival Tolkien, Pratchett, and Rowling in fantasy?
>
> Compared to that, software is in a pretty decent state. Being finally in an
> US managed company now (after a merger), I can see why: Way too impatient.
> Good products take time. If you take the time away, you get nothing. Not
> just bad products, no products at all. Business classes often ask why
> software is always late and over budget? Answer: Because the schedule is
> always too aggressive and the budget grossly understated. Unfortunately,
> this answer is only written in Dilbert, not in any MBA program.
IMHO, the reason for understating the schedule & cost is simply that
if the cost & schedule were correctly state up front, the product
would be cancelled by management before it got started.
It becomes a vicious circle.
Kai
--
Kai Harrekilde-Petersen <khp(at)harrekilde(dot)dk>
Robert Myers
> in response to Robert Myers:
>
> Sure. But software of and by itself can never hurt someone physically. A
> device that includes software as a major component can do that. I'm
> trying to remember software that has lead to deaths. Therac-25; possibly
> AirFrance 447 - anything else? Pretty good record, I'd say.
>
What you really should be saying here is that the harm caused by
software, which surely includes many deaths, usually can't be traced
directly to software. When people become frustrated and irritable
because of delays (the network is down, the file is lost or corrupted,
a messy bureaucratic error is made, and on and on and on), they make
more mistakes. People under stress they can't control are at elevated
risk of cardiovascular events. Equipment or medication doesn't arrive
on time.
Doctors are overworked. People become aggressive in traffic, and so
on. People do die.
If lawyers could figure out how to get their hands on a piece of that
action, they would, but they don't seem to be able to get their arms
around it. In any case, software is sold "as is," a circumstance to
which your employer was a big contributor, thus adding an important
cost of entry for the legal profession to try
anything at all.
Our very complicated society is increasingly dependent on complicated,
networked software. The public in general is blase about it, so blase
that that you have a know-it-all writing articles for the Harvard
Business Review like "IT Doesn't Matter." It does matter, and the
problems of the industry, which only grow with time do matter.
We don't *have* to be carting people around in airplanes. We *have*
to be using computers.
> > Commercial and military aircraft involve a degree of design
> > sophistication that is, in itself, breathtaking. What is even more
> > amazing is the large numbers of different disciplines that must work
> > together successfully to produce a product that works and can be sold
> > into a market with razor thin (or, in some cases, negative) margins.
> > Boeing tried to shop out a lot of work on the Dreamliner and
> > discovered what should have been no surprise to anyone: offshoring
> > critical parts across town is as hard as offshoring to Taiwan, Mexico,
> > or India. It's a measure of how hard it is to get it right and that,
> > for all the criticism they endure, they do know how to get it right.
>
> Really? Looking at the aviation industry in general, I find it has a
> pretty poor record of designs. They've been designing planes for ~100
> years now [compared to ~50 years for software], the design-space is
> small compared to software, they use longer design cycles and larger
> teams, and they still don't get it right.
>
I could have talked about microprocessors, which could more fairly be
described as my golden child. I chose to talk about an industry the
workings of which I have more direct knowledge. Microprocessors and
software share a common trait that airplanes do not: they can be
accurately simulated in detail. Microprocessors share with airplanes
the feature that some of the design is inevitably cut and try, and
each cut and try is very costly. No simulation of airplanes at the
level at which microprocessors are simulated is currently possible.
And the design space is always growing.
Software: complete simulation possible, cut and try is inexpensive
Microprocessors: nearly complete simulation is possible, cut and try
is expensive.
Airplanes: complete simulation is not possible, cut and try is very
expensive (and involves risk to life).
> Its also arguable that planes are less complex than high-complexity
> software (e.g. OSes). This was definitely so until the fly-by-wire and
> dynamically-unstable aircraft - compare the number of parts in a DC-10
> to the number of lines of code in OS/360. And then consider the number
> of design flaws in the DC-10.
>
Software, and, in particular, OS software, is complex because it
chooses to be. The industry always chooses speed and size of feature
set over transparency and reliability.
Microsoft operates with huge margins. Life may be much harder for
others, perhaps even including IBM (although I really doubt it). The
design of a long haul modern commercial airplane is a bet-the-company
proposition. Microsoft screwed up Vista and Intel screwed up Itanium
*and* NetBurst. IBM fumbled the microprocessor revolution. Lockheed
and McDonnell-Douglas are both out of the commercial airplane
business. Microsoft, Intel, and IBM are doing just fine. Of course,
DEC, CDC and countless others are all out of business, but none had a
business history remotely comparable to the big players in aviation.
As it is, neither a Boeing nor an Airbus can afford all the investment
risk. Big hunks are laid off on subcontractors, especially the
manufacturers of jet engines, who generally sell at a negative
margin. Can you imagine Microsoft, Intel, or IBM selling at a
negative margin? To be fair, the game box market is a lot like the
jet engine business, as at least Sony and Microsoft sell at negative
margins, or close to it, and Sony has, I'm sure, become aware of the
risks of bet-the-company propositions. To be fair, OS/360 was a bet-
the-company proposition for IBM, but that was a long time ago.
> > Compare any of this to the state of software. Licensing? That's a
> > good one. Liability? Bring your reading glasses when buying
> > software, because the number of rights you give up when you buy
> > anything is the only thing growing faster than new security exploits.
> > Management and product integration? Look at the schedule slippage and
> > design of the user interface for Vista.
>
> As opposed to the latest efforts from Airbus (380 2+ years?), Boeing
> (787 2years+?), Lockheed-Martin (F-22 N-years)?
>
So far as I know, the Dreamliner slipped mostly because of a decision
about outsourcing, not because of design issues. Military aircraft
are operating in flight regimes that weren't even possible not so long
ago, and the Pentagon is notorious for changing priorities and
requirements.
> BTW: The F-22 design RFP was issued in 1986 (and that was preceded by
> the ATF requirement spec in 1981, so I'm sure design work started
> earlier). It took them till 1990/91 to deliver the demo-prototypes. The
> first production version was delivered in 2003. So:
> - 1981 for initial spec
> - 1986 for formal RFP
> - 1990 for demo
> - 2003 for production
> You really think that aviation does a better job than software?
>
Yes. If aviation operated the way software does, airplanes would be
crashing constantly, just as software is crashing constantly.
> In case you think that this is peculiar to military/DoD, consider the
> Airbus A380.
> - 1988: prelim design work
> - 1990: formal project announce
> - 1994: A3XX design started
> - 2000: A380 started
> - 2005: prototype
> - 2007: first delivery
> Not much better than the F-22, is it?
I have no direct exposure to the European aviation business. Airbus
operates in a completely different financial/political/labor/
managerial environment from any of the US businesses we have
discussed.
>
> > If you're a "luser" and not a part of the scam, you get used to things
> > never quite working the way they're supposed to, debugging practically
> > every new product you use, and now an endless procession of updates to
> > fix security holes, the plethora of which updates is a constant
> > distraction and annoyance.
>
> Seen the list of FAA advisories/airworthiness directives. A quick visit
> to FAA site reveals 71 directives in the past 60 days. Do you really
> think that this reveals rock-solid engineering?
>
This comment is jarring. Airplanes fly for decades through all kinds
of conditions. Tires have to be replaced with every few takeoffs and
landings. Maintenance and even inspection are iffy and sometimes
outright illegal and fraudulent. Federal oversight and even DoD
maintenance discipline is weak. Do you know when the last B-52 was
built, never mind designed?
Mostly, though, engineers can't engineer based on science that isn't
yet known. Fatigue and fracture, especially of new materials, isn't
all that well understood. Tribology (the science of wear) is an
active field of research. What computer program can predict that a
particular longeron will start failing after years of of apparently
satisfactory service? One fails, a jet crashes, and all of the
corresponding parts should be examined for similar problems. That's
*good* engineering practice, not bad.
Despite the age of the underlying disciplines that explore the science
behind the engineering, no one that I know of imagines that any of it
is complete.
> Try comparing designs of like complexity: in general, bridges and cars
> (two other examples you've used) are really simple compared to complex
> software such as, oh, operating systems.
>
What the engineers can include in their notions of complexity is
limited by the underlying science (and, most notably, by the size and
capabilities of the computers that can be brought to bear). The
complexity of software is self-inflicted. One could argue, and some
have, that the rapid adoption of composites in airplanes is a self-
inflicted risk. It *is* a gamble. If it goes bad, though, there will
be hell to pay.
> As an exercise, figure out how many parts there are in a car. Then, find
> a program with roughly the same complexity (maybe same number of LOC?).
> Compute the man-years spent developing both. Count the number of recalls
> for the car and the number of design flaws fixed. Compare that against
> the number of bugs in the equivalent program. You might be a little
> surprised.
Such comparisons are meaningless. How many parts does a jackscrew,
the failure of which in an airplane can be catastrophic and the result
of many, many different things, count for? The physics of something
so humble as a tire are mind-boggling. How many parts does a
squirming, hot, underinflated tire count for, never mind how it
interacts with the entirety of the vehicle?
Your response is discouraging.
Robert.
Anne & Lynn Wheeler
Robert Myers <rbmye...@gmail.com> writes:
> bet-the-company propositions. To be fair, OS/360 was a bet-
> the-company proposition for IBM, but that was a long time ago.
so was future system (which was going to completely replace 360) ... but
was canceled without ever being announced ... some past posts mentioning
future system
http://www.garlic.com/~lynn/submain.html#futuresys
fergus/morris book make some claims that it took more than
20 yrs to recover ... recent reference
http://www.garlic.com/~lynn/2009g.html#0
there were also claims that if it had been any company, they wouldn't
have recovered.
in the 80s, I had sponsored Boyd's briefings at IBM ... he had been head
of lightweight fighter plane design at the pentagon ... claimed credit
for cutting weight of f15 in half (and significant improvement in f18)
... and responsible for much of f16 design. f20/tigershark also showed
a lot of his influence/philosophy ... being significantly cheaper and
less complicated than f16, much lower skill level to maintain and much
higher ratio of flt hrs to maintenance hrs. There were claims that
f20/tigershark fell to heavy lobbying and political influence (from more
profitable programs).
boyd had done a 1970 tour in charge of spook base ... there was some
reference to it having been a $2.5B windfall for IBM ... which would
have contributed to IBM being able to survive future system. misc. past
posts mentioning Boyd
http://www.garlic.com/~lynn/subboyd.html#boyd
boyd has also been credited with battle plan for desert storm and there
have been comments that a major problem going into current conflicts was
that boyd had died in 1997.
this is something more recent that Boyd would have been in the middle of
(if he was still around) regarding drones ...
http://www.theregister.co.uk/2009/04/29/young_usaf_predator_pilot_officer_slam/
--
40+yrs virtualization experience (since Jan68), online at home since Mar1970
Robert Myers
> Mayan Moudgill wrote:
> > Robert Myers wrote:
>
> >> The state of software is a national disgrace.
>
> > I don't agree. I actually think software is in pretty decent state.
>
> > What yardstick are you using? What other product design activity would
> > you compare it against?
>
> In the USA? I'd have troubles. Housing? Ikea-style "cardboard" houses
> with slits under the window so big that you don't need doors¹ ;-). Cars?
> With technology as developed in the 60s, just larger? Financial "products"
> which rival Tolkien, Pratchett, and Rowling in fantasy?
>
architect is paid in part to oversee construction. In practice, most
houses are built without architect oversight.
Financial engineering and "science" is a different story. I can't
think of a more egregious example of overreaching based on pseudo-
knowledge. Much of that pseudo-knowledge can be traced to GIGO
computing.
> Compared to that, software is in a pretty decent state. Being finally in an
> US managed company now (after a merger), I can see why: Way too impatient.
> Good products take time. If you take the time away, you get nothing. Not
> just bad products, no products at all. Business classes often ask why
> software is always late and over budget? Answer: Because the schedule is
> always too aggressive and the budget grossly understated. Unfortunately,
> this answer is only written in Dilbert, not in any MBA program.
>
Engineers in other disciplines haven't been so supine. Standards for
engineering practice are controlled by engineering societies, not by
MBA's and suck-up managers.
Boston recently had a major public works project in which there was
much malfeasance and misfeasance:
http://en.wikipedia.org/wiki/Big_Dig
Some of the incidents raise real questions about the practice of
construction engineering, at least in my mind. On the other hand,
does any society connected with computers do anything like this:
http://www.asce.org/reportcard/index.cfm?reaction=states&page=MA
> Don't get me wrong: European managers get the schedule and the budget just
> as badly wrong as US managers - there is all way too much wishful thinking
> involved. But my experience is that they listen more to their engineers and
> finally bite the bullet, and let the project slip and the budget increase.
> Of course, European managers also pull the plug when a project takes too
> long to achieve the desired quality.
>
> ¹) People often say "when houses where build like software, you'd get..."
> Well, you'd get houses like they are. With patched roofs and windows that
> open only when you kick them. With open (back)doors for all kind of vermin.
>
There's lots of shoddy construction, but that's because the business
and oversight of the business are so fragmented and localized.
Robert.
Mayan Moudgill
>
> I could have talked about microprocessors, which could more fairly be
> described as my golden child.
Microprocessors are much less complex than complex system software. (And
yes, I've done both, so I know what I'm talking about).
I chose to talk about an industry the
> workings of which I have more direct knowledge. Microprocessors and
> software share a common trait that airplanes do not: they can be
> accurately simulated in detail.
Sorry, you're talking about a subset of all software. Consider parallel,
or worse, distributed programs. They are less simulatable than
microprocessors, and may even be less simulatable than an airplane.
Consider simulating a program running on a few 100 machines on 2 or 3
different processors and operating systems, linked using several levels
of routers, running a MPI program. Think you can really simulate that
with any real fidelity?
Mayan Moudgill
>>
>
> So far as I know, the Dreamliner slipped mostly because of a decision
> about outsourcing, not because of design issues.
>
Really? How about the wingbox issue (6 month delay)? And the redesign
work to get rid of the overweight issue [which is still continuing]?
Its convenient to blame all the delay on outsourcing, and if they had
done more development inhouse, they might have caught the problem
earlier, but the fact still remains that they had to do redesign.
Mayan Moudgill
> On Aug 27, 10:35 am, Mayan Moudgill <ma...@bestweb.net> wrote:
>
>>in response to Robert Myers:
>
>
>
>>Sure. But software of and by itself can never hurt someone physically. A
>>device that includes software as a major component can do that. I'm
>>trying to remember software that has lead to deaths. Therac-25; possibly
>>AirFrance 447 - anything else? Pretty good record, I'd say.
>>
>
> What you really should be saying here is that the harm caused by
> software, which surely includes many deaths, usually can't be traced
> directly to software. When people become frustrated and irritable
> because of delays (the network is down, the file is lost or corrupted,
> a messy bureaucratic error is made, and on and on and on), they make
> more mistakes. People under stress they can't control are at elevated
> risk of cardiovascular events. Equipment or medication doesn't arrive
> on time.
> Doctors are overworked. People become aggressive in traffic, and so
> on. People do die.
Sure lots of stuff stresses people, leading to mistakes. Better
human-factor design can be used to lessen the burden on the user.
Unfortunately, the aircraft industry has a particularily bad history in
this area. Cockpit alarm/guage placement has been a contributory factor
in many "pilot error" plane crashes. And this despite the fact that the
average pilot undergoes many, many hours of training to deal with the
user-interface.
Robert Myers
The main point here is that the problem was caught before anything
shipped. Subcontracting the manufacture of wingboxes would not have
been novel for an airplane manufacturer, so this is one problem that
can't fairly be blamed on outsourcing aggressiveness.
Every industry makes mistakes, sometimes big ones and sometimes big
ones that result in deaths. Failure patterns that should have been
examined more closely are sometimes ignored until it's too late.
Murphy's Law applies everywhere.
Nevertheless, I claim that software is different. Means already exist
to make software more maintainable, less buggy, and more transparent.
They simply aren't used because there is no incentive to do so. The
practice of the industry favors farming the risk out--to the end
buyer.
Robert.
Robert Myers
The pilot is a specialist. It's his job to cope with all kinds of
things. No one *has* to get on an airplane. Everyone who does
accepts a risk that could be calculated and insured against.
The realities of software, in contrast, challenge the capabilities and
the patience of ordinary citizens every day, no insurance can be
procured against such a diffuse risk, and I'm certain that the cost to
society dwarfs the cost of airplane accidents due to pilot error.
Robert.
Robert Myers
> Robert Myers wrote:
>
> >
> > I could have talked about microprocessors, which could more fairly be
> > described as my golden child.
>
> Microprocessors are much less complex than complex system software. (And
> yes, I've done both, so I know what I'm talking about).
>
concurrent software, could be eliminated by using hardware design
technology that specifically accounts for concurrency.
> I chose to talk about an industry the
> > workings of which I have more direct knowledge. Microprocessors and
> > software share a common trait that airplanes do not: they can be
> > accurately simulated in detail.
>
> Sorry, you're talking about a subset of all software. Consider parallel,
> or worse, distributed programs. They are less simulatable than
> microprocessors, and may even be less simulatable than an airplane.
>
> Consider simulating a program running on a few 100 machines on 2 or 3
> different processors and operating systems, linked using several levels
> of routers, running a MPI program. Think you can really simulate that
> with any real fidelity?
Yes, indeed, and I've been wailing about this for years. Why does
anyone expect anything to work? Reading any discussion here about
locks and such is a hair-raising experience. Nothing will change so
long as incalculable risk can be pushed off onto the end user.
Robert.
Andrew Reilly
> Nevertheless, I claim that software is different.
To clarify: you're pretty much talking about "software as a product"
here, right? Software-as-a-component of a system has historically, as
far as I can see, had about the same sort of quality control and
resulting quality *and legal liability* as the rest of the system in
question.
So: it's not software per-se that is at fault. What do you think makes
software-as-a-product different?
> Means already exist
> to make software more maintainable, less buggy, and more transparent.
> They simply aren't used because there is no incentive to do so.
You keep saying that, but it simply isn't so.
Just as another data point, I came across ohloh today, and they show this
graph:
http://www.ohloh.net/languages/compare?
measure=loc_changed&percent=&l0=c&l1=csharp&l2=cpp&l3=java&l4=javascript&l5=-1&l6=-1&commit=Update
That seems to show to me quite clearly that the amount of new and
maintenance work being done in C has been falling steadily since 2000, in
favour of Java and JavaScript (and a bunch of others too small to see
compared to these). C++ use looks to be fairly stable, and that is
perhaps a problem. [And that's all for software-as-a-product (free)
projects. Doesn't measure software-as-a-component, as far as I can see.]
> The
> practice of the industry favors farming the risk out--to the end buyer.
Perhaps that's because the buyers accept the risk in preference to paying
a higher cost? You bang on about everyone having to use the internet for
everything, but that's only because everyone's decided that they can. Up
until a few years ago all of the PCs and all of the software on them were
a joke, and no-one really expected them to do anything sensible. Things
that had to be reliable were written on pieces of paper. The internet
was written as an experiment, a play-thing, by students and researchers.
It is what it is. If you want reliable and dependable and professionally
developed, look to the telephone network. And be prepared to pay
telephone network rates.
Cheers,
--
Andrew
Robert Myers
users.org> wrote:
> On Thu, 27 Aug 2009 17:19:09 -0700, Robert Myers wrote:
> > Nevertheless, I claim that software is different.
>
> To clarify: you're pretty much talking about "software as a product"
> here, right? Software-as-a-component of a system has historically, as
> far as I can see, had about the same sort of quality control and
> resulting quality *and legal liability* as the rest of the system in
> question.
>
> So: it's not software per-se that is at fault. What do you think makes
> software-as-a-product different?
>
will be used. Software-as-a-product (by itself or even sold with a
computer system) is a tool. No manufacturer of tools can foresee all
possible ways the tool can be used. If you use the tool improperly
and injure yourself, that's your problem. If you use the tool
properly but injure yourself, anyway, that may or may not be the
manufacturer's problem.
Rydell is currently being sued because it doesn't put warnings about
*heat stroke* on *football helmets and pads*. Imagine if the world of
software worked that way. Is wearing a football helmet in hot, humid
summer weather a proper use of that equipment? If not, whose
responsibility is it to warn of foreseeable danger?
In the case of software, your sole recourse is to return the tool and
usually not even for a refund. That's what's different. It all goes
back to IBM and its idea of "licenses," which were then magically
turned into shrink-wrap licenses, which all but eliminated seller
liability when you opened the package.
You have no reasonable expectation of anything, and courts have upheld
that position on the part of vendors. If software is embedded in
something other than a computer, there is no equivalent of a shrink
wrap license. In the case of a computer, you may have a
manufacturer's warranty, but it won't help you if the software they
sold you is only as good or as bad as the software sold in a shrink-
wrap version.
> > Means already exist
> > to make software more maintainable, less buggy, and more transparent.
> > They simply aren't used because there is no incentive to do so.
>
> You keep saying that, but it simply isn't so.
>
There is apparently disagreement on this point. I see no use in
arguing it further.
> Just as another data point, I came across ohloh today, and they show this
> graph:
>
> http://www.ohloh.net/languages/compare?
> measure=loc_changed&percent=&l0=c&l1=csharp&l2=cpp&l3=java&l4=javascript&l5=-1&l6=-1&commit=Update
>
> That seems to show to me quite clearly that the amount of new and
> maintenance work being done in C has been falling steadily since 2000, in
> favour of Java and JavaScript (and a bunch of others too small to see
> compared to these). C++ use looks to be fairly stable, and that is
> perhaps a problem. [And that's all for software-as-a-product (free)
> projects. Doesn't measure software-as-a-component, as far as I can see.]
>
I don't know why you think of java as some kind of magic fix. To me,
the different versions of java have been more than just a minor
nuisance. Admittedly, I'm not a java jockey, and I see little use in
becoming one.
To me, java is just another problem. Okay, it eliminates some common
errors of c. What else?
Some things are written in java, some in c, some in c++, some in c#,
some in ruby, some in python, some in perl. Lots of the software I
use is still written in Fortran. I've never heard anyone mention any
of those languages as particularly desirable for SMP applications.
Rather, I hear about languages like Eiffel, Ada, Occam, or Haskell.
Yet here you are addressing me as if I were some kind of stubborn fool
because I don't see that it was all settled years ago when Sun
introduced Java. Are you truly serious?
I don't have a dog in this fight, and I don't see a clear winner.
What I see is what I've commented on: total, unmitigated, inexcusable
chaos.
> > The
> > practice of the industry favors farming the risk out--to the end buyer.
>
> Perhaps that's because the buyers accept the risk in preference to paying
> a higher cost? You bang on about everyone having to use the internet for
> everything, but that's only because everyone's decided that they can. Up
> until a few years ago all of the PCs and all of the software on them were
> a joke, and no-one really expected them to do anything sensible. Things
> that had to be reliable were written on pieces of paper. The internet
> was written as an experiment, a play-thing, by students and researchers.
> It is what it is. If you want reliable and dependable and professionally
> developed, look to the telephone network. And be prepared to pay
> telephone network rates.
This isn't a few years ago. Land lines are disappearing faster than
biodiversity in the Amazon rain forest. The entire infrastructure of
our technological civilization either is or soon will be riding on
packet-switched networks.
Robert.
Andrew Reilly
> Rydell is currently being sued because it doesn't put warnings about
> *heat stroke* on *football helmets and pads*. Imagine if the world of
> software worked that way. Is wearing a football helmet in hot, humid
> summer weather a proper use of that equipment? If not, whose
> responsibility is it to warn of foreseeable danger?
I remember reading, long ago, about the last American motorcycle helmet
manufacturer pulling up stumps because they couldn't stand the cost of
litigation. Not because they were making bad helmets, but because they
were the only ones around after an accident with what looked like deep
enough pockets to support the hospital bills. Litigation isn't always
the right answer. Having to warn about heat stroke on helments is like
warning against drying the cat in the microwave. In a sane society
neither would be necessary. I certainly don't want to live in a society
where my software providers have to explicitly warn against all of the
non-uses any idiot might put it to, or to be liable for those misuses.
> I don't know why you think of java as some kind of magic fix.
I don't. Don't particularly like it, myself.
> To me,
> the different versions of java have been more than just a minor
> nuisance. Admittedly, I'm not a java jockey, and I see little use in
> becoming one.
>
> To me, java is just another problem. Okay, it eliminates some common
> errors of c. What else?
That's just it. It *eliminates* the entire class of software errors
(buffer overruns and attendent stack crashes) that you have been saying
is the big devil of the state of software, in both this and the SoftBound/
HardBound thread. It is more popular (in that more software is being
written in it, and more jobs are advertised for it) than C, and
increasingly so. To me that reads that most people doing software care
about taking steps to avoid that class of software errors.
To keep saying "no one is doing anything about it", as you are, without
some sort of metric or substance is just so much wind.
> Yet here you are addressing me as if I were some kind of stubborn fool
> because I don't see that it was all settled years ago when Sun
> introduced Java. Are you truly serious?
I'm sorry if I've given the impression that Java is the be-all and end-
all of languages. It clearly isn't. You do seem to be quite stubborn
(indeed shrill) in your insistance that "nothing is changing" and that we
are engaged in some kind of catastrophy (that is "software's fault").
> I don't have a dog in this fight, and I don't see a clear winner. What I
> see is what I've commented on: total, unmitigated, inexcusable chaos.
I have no answer to that. Clearly our views of the world differ.
> This isn't a few years ago. Land lines are disappearing faster than
> biodiversity in the Amazon rain forest.
to be replaced by the mobile phone network, also by the telcos, and also
substantially a product of software that works.
> The entire infrastructure of
> our technological civilization either is or soon will be riding on
> packet-switched networks.
So? Are you concerned by the fact that all of the major operating system
platforms in use today substantially pre-date the consumer use of these
packet-switched networks? Oh, no: legacy code!
Cheers,
--
Andrew
Robert Myers
>
> > To me, java is just another problem. Okay, it eliminates some common
> > errors of c. What else?
>
> That's just it. It *eliminates* the entire class of software errors
> (buffer overruns and attendent stack crashes) that you have been saying
> is the big devil of the state of software, in both this and the SoftBound/
> HardBound thread. It is more popular (in that more software is being
> written in it, and more jobs are advertised for it) than C, and
> increasingly so. To me that reads that most people doing software care
> about taking steps to avoid that class of software errors.
>
I said in the SoftBound/HardBound thread that it seemed like a lot of
work that would fix only a subset of the problems. If someone is
willing to do it, though, it's better than nothing. I don't see the
17 gigabytes of Debian code I just downloaded being rewritten mostly
in anything other than c any time soon.
> To keep saying "no one is doing anything about it", as you are, without
> some sort of metric or substance is just so much wind.
>
The metric I use is the endless haggling anywhere you go about the
wonders of whatever software scheme someone is trying to shill.
Languages, language enhancements, and IDE's multiply like rabbits,
while something like ada languishes. I conclude that the entire
industry suffers from ADHD. If that seems shrill, I'm sorry. Well,
actually, I'm not really all that sorry. I've listened to too much
horse manure from snotty software jocks. I thought Ruby was really
cool, a lot more cool than Java, but it's already being replaced by
something else even more cool, the name of which I haven't bothered to
try to remember.
When I download a source tarball, my stomach tightens. I'm going to
install Debian because a particular software package I want to use
badly will run out of the box on it. I don't have time to hack my way
through a jungle of dependencies written in a veritable Babel of
languages.
The shrill tone, to be honest, comes from the shrillness of people in
the industry who think everything is just wonderful. Maybe it is if
you're in some niche somewhere, surrounded by a corporate culture that
does everything a certain way and hidden behind a human-tended
firewall. For the ordinary citizen, it's like the wild west.
Here's a random pick from slashdot firehose:
http://www.washingtonpost.com/wp-dyn/content/article/2009/08/26/AR2009082601694.html
> > Yet here you are addressing me as if I were some kind of stubborn fool
> > because I don't see that it was all settled years ago when Sun
> > introduced Java. Are you truly serious?
>
> I'm sorry if I've given the impression that Java is the be-all and end-
> all of languages. It clearly isn't. You do seem to be quite stubborn
> (indeed shrill) in your insistance that "nothing is changing" and that we
> are engaged in some kind of catastrophy (that is "software's fault").
>
I think we are headed for big trouble. Some of us have a
psychological need to have our own sky-is-falling scenario, and I
don't want to be on the same bandwagon with Al Gore. Not only do I
not think things are getting better, I think they are getting worse.
It isn't, by far, *all* the fault of software, but software is a huge
bottleneck. WHO is going to wade through IPv6, and WHEN?
> > I don't have a dog in this fight, and I don't see a clear winner. What I
> > see is what I've commented on: total, unmitigated, inexcusable chaos.
>
> I have no answer to that. Clearly our views of the world differ.
>
Yes.
> > This isn't a few years ago. Land lines are disappearing faster than
> > biodiversity in the Amazon rain forest.
>
> to be replaced by the mobile phone network, also by the telcos, and also
> substantially a product of software that works.
>
When I think of telcos, I think of "RBOC's," which are a US
phenomenon. They are regulated monopolies and they are moving toward
extinction, so far as I can tell, to be replaced by a far less
regulated and far more predatory wireless industry. Whatever it may
be like in the rest of the world, in the US they will be nothing like
the telcos of old.
> > The entire infrastructure of
> > our technological civilization either is or soon will be riding on
> > packet-switched networks.
>
> So? Are you concerned by the fact that all of the major operating system
> platforms in use today substantially pre-date the consumer use of these
> packet-switched networks? Oh, no: legacy code!
>
http://www.makelinux.net/kernel_map
Robert.
Kim Enkovaara
> Houses are not built by architects and engineers. In theory, an
> architect is paid in part to oversee construction. In practice, most
> houses are built without architect oversight.
That depends on the country. Here in northern europe the winters
are cold enough that the house building is quite regulated.
The plans have to be approved to fullfill the building code,
the construction site has to have building inspector and foreman with
certain legal liabilities during the whole project. And before the
building is accepted for living it goes trough inspection done by
municipal inspector.
--Kim
Robert Myers
In theory, this is all true in most places in the US. The town I live
in, which is relatively wealthy, has only one part-time building
inspector. Many of the homes here *are* built with architect
oversight, and the building codes are quite strict, but this is not a
normal town.
I saw some really shoddy construction when I lived in Southern
California.
Robert.
Carlie Coats
Or the building-inspections department becomes a "captured" fiefdom
that protects existing firms, with rules more designed to keep new
competition out of the area, and that provide opportunities for the
inspectors to engage their lust for power.
And you see lots of that last, lust for power -- dozens of little
tin-pot Hitlers abound.
That's what I see here in NC, USA.
An interesting, off-topic aside: G.F. Handel was a good friend of
George III, and wrote _Judas Maccabeus_ for him in the 1760s. The
oratorio is just filled with thoughts of liberty; it opens with a
solo by the young general, "Let not lust for unbounded power my
heart inspire." Rumor has it the oratorio met with approval across
the Atlantic in the Colonies; one has to wonder just how much trouble
George Friederich was creating for his good friend a decade later!
FWIW.
-- Carlie Coats
Del Cecchi
"Kim Enkovaara" <kim.en...@iki.fi> wrote in message
news:CxKlm.6021$La7....@uutiset.elisa.fi...
Pretty much the same thing here in Northern USA. The big problems
come when standards change and designs and building practices don't
catch up, ie the problems with moisture and mold caused by new tighter
building standards without correct details about ventilation and
construction practices.
del
Robert Myers
>
> Or the building-inspections department becomes a "captured" fiefdom
> that protects existing firms, with rules more designed to keep new
> competition out of the area, and that provide opportunities for the
> inspectors to engage their lust for power.
>
> And you see lots of that last, lust for power -- dozens of little
> tin-pot Hitlers abound.
>
> That's what I see here in NC, USA.
>
> An interesting, off-topic aside: G.F. Handel was a good friend of
> George III, and wrote _Judas Maccabeus_ for him in the 1760s. The
> oratorio is just filled with thoughts of liberty; it opens with a
> solo by the young general, "Let not lust for unbounded power my
> heart inspire." Rumor has it the oratorio met with approval across
> the Atlantic in the Colonies; one has to wonder just how much trouble
> George Friederich was creating for his good friend a decade later!
>
> FWIW.
At least part of the explanation for the Wild West/Robber Baron aspect
of the industry has to come from the time in which the industry grew
so rapidly. Microcomputers were coming on line as Ronald Reagan took
office and suspicion of regulation and government in general were
advancing like a tidal wave. We are all (in the US, at least) now
living through the after-effects of aggressive deregulation.
I'd like to hope that the idea that completely unregulated free
markets always produce optimal outcomes has now been thoroughly
debunked, but I fear that it will take calamity on the scale of the
collapse of Wall Street for people to wake up to the fact that
computers are as central now to civilization as finance.
Robert.
Rob Warnock
+---------------
| Bernd Paysan <bernd....@gmx.de> writes:
| > ...US managed company now (...), I can see why: Way too impatient.
| > Not just bad products, no products at all. Business classes often ask why
| > software is always late and over budget? Answer: Because the schedule is
| > always too aggressive and the budget grossly understated. Unfortunately,
| > this answer is only written in Dilbert, not in any MBA program.
|
| IMHO, the reason for understating the schedule & cost is simply that
| if the cost & schedule were correctly state up front, the product
| would be cancelled by management before it got started.
Indeed. I have some 1970's experience with embedded software that
simply *had* to be right[1], and I learned two things: (a) It is
actually possible to develop software that is essentially "correct"
[bug-free]; and (b) it costs 3-5 *times* as much as "normal" coding
practice to do so.
There are very few markets which will bear the cost of "bug-free" code.
[But there *are* a very few...]
-Rob
[1] DCA (Atlanta), with the SmartMux 200 series of products.
The code was in socketed UVPROM, but the company was so
cash-strapped [and the large UVPROMs we were using at the
time so expensive] that we literally couldn't *afford* to
replace all the customers' ROMs to correct a major bug!
We simply *had* to figure out how not to have any major bugs,
so we did.
-----
Rob Warnock <rp...@rpw3.org>
627 26th Avenue <URL:http://rpw3.org/>
San Mateo, CA 94403 (650)572-2607
Bernd Paysan
> I'd like to hope that the idea that completely unregulated free
> markets always produce optimal outcomes has now been thoroughly
> debunked, but I fear that it will take calamity on the scale of the
> collapse of Wall Street for people to wake up to the fact that
> computers are as central now to civilization as finance.
I can't see a free, unregulated market in software, or computers in
general, as long as patents and copyright are valid. Note that the
proponents of free markets everywhere now include patents and copyrights
into the market-stifling regulations, as well ;-). And they use free
software as examples that it works without such monopolies.
And remember what the problem of the financial collapse is: banks "too
big to fail". A free market works against big companies. If the
companies are big, it's a sign that the market is - for one or another
reason - not free. Take the first two domino stones of the crisis:
Fanny Mae and Freddy Mac. Are these banks that work in a free market?
No, they are supported by the state to give cheaper mortgages than
normal banks could.
For similar reasons, the banks that got most into calamities in Germany
were the Landesbanken (owned by the countries), and the DEPFA, which is
now part of HRE, but really is just a privatized, formerly state-owned
bank (with about the same purpose as Fanny Mae). So the free-market
guys still have some sort of argument. And the regulation didn't work
because of conflicts of interest.
Note that free markets still need rules. Otherwise, the "might is
right" doctrine rules.
Nicholas King
> Kai Harrekilde-Petersen <k...@harrekilde.dk> wrote:
> +---------------
> | Bernd Paysan <bernd....@gmx.de> writes:
> | > ...US managed company now (...), I can see why: Way too impatient.
> | > Good products take time. If you take the time away, you get nothing.
> | > Not just bad products, no products at all. Business classes often ask why
> | > software is always late and over budget? Answer: Because the schedule is
> | > always too aggressive and the budget grossly understated. Unfortunately,
> | > this answer is only written in Dilbert, not in any MBA program.
> |
> | IMHO, the reason for understating the schedule & cost is simply that
> | if the cost & schedule were correctly state up front, the product
> | would be cancelled by management before it got started.
> +---------------
>
> Indeed. I have some 1970's experience with embedded software that
> simply *had* to be right[1], and I learned two things: (a) It is
> actually possible to develop software that is essentially "correct"
> [bug-free]; and (b) it costs 3-5 *times* as much as "normal" coding
> practice to do so.
>
> There are very few markets which will bear the cost of "bug-free" code.
> [But there *are* a very few...]
software , along with maintenance and the operational costs on the user?
If we consider the difference in cost across the whole system between
*doing it right* rather than *doing it cheap* a lot of the time it makes
sense to *do it right* the first time.
However there are quite a few reasons why it's not done. We get things
like people being overly optimistic about risk , the real costs not
being borne by the developer but by the user and markets being created
by support as well vendor lock-in along with inertia (people prefer to
do things the same way rather than learn new procedures (or correct
procedures)). Then we have politics as well as the old laziness.
Cheers,
Nicholas King
Rob Warnock
+---------------
| Rob Warnock wrote:
| > Indeed. I have some 1970's experience with embedded software that
| > simply *had* to be right[1], and I learned two things: (a) It is
| > actually possible to develop software that is essentially "correct"
| > [bug-free]; and (b) it costs 3-5 *times* as much as "normal" coding
| > practice to do so.
| >
| > There are very few markets which will bear the cost of "bug-free" code.
| > [But there *are* a very few...]
|
| Is the 3-5 *times* cost factor including the full cost of the developed
| software , along with maintenance and the operational costs on the user?
|
| If we consider the difference in cost across the whole system between
| *doing it right* rather than *doing it cheap* a lot of the time it makes
| sense to *do it right* the first time.
I'm not sure what you mean by "operational costs" here, since I was
speaking of embedded software that a user doesn't so much "run" as
merely "turn on" [when they turn on the box that contains it].
But my "3-5 times" swag [which, by the way, was just a rough estimate
based on my recollection of events from ~35 years ago, not any
kind of formal academic study] was meant to speak just to initial
development/coding costs of code that "just had to work". You are
correct that some of that premium could be discounted by lessened
maintenance costs, but I wasn't addressing that per se.
+---------------
| However there are quite a few reasons why it's not done. We get things
| like people being overly optimistic about risk , the real costs not
| being borne by the developer but by the user and markets being created
| by support as well vendor lock-in along with inertia (people prefer to
| do things the same way rather than learn new procedures (or correct
| procedures)). Then we have politics as well as the old laziness.
+---------------
All of that is typical these days, as you note. I was speaking only
of one extreme case in which the company's life itself was on the line
[literally -- for a while there we were so cash-tight that if we'd had
to ship updated UVROMs to all our customers we wouldn't have been able
to make the next payroll!], so we found a way to *make* it happen! ;-}
Such extreme circumstances are quite rare, outside of human-safety-critical
codes -- certain medical devices, aviation fly-by-wire codes, and the like.
-Rob
Anne & Lynn Wheeler
rp...@rpw3.org (Rob Warnock) writes:
> But my "3-5 times" swag [which, by the way, was just a rough estimate
> based on my recollection of events from ~35 years ago, not any
> kind of formal academic study] was meant to speak just to initial
> development/coding costs of code that "just had to work". You are
> correct that some of that premium could be discounted by lessened
> maintenance costs, but I wasn't addressing that per se.
I've used "4-10" times to take a "well tested" application and turn it
into a (industrial/business strength) service.
we had been doing the ha/cmp product ... some old posts
http://www.garlic.com/~lynn/subtopic.html#hacmp
part of it was high-availabiilty and part was cluster
scaleup ... some old email
http://www.garlic.com/~lynn/lhwemail.html#medusa
this post references a jan92 meeting discussing scaleup
http://www.garlic.com/~lynn/95.html#13
and shortly after the jan92 meeting, the scaleup part was transferred
and we were told we couldn't work on anything with more than four
processors.
two of the people mentioned in the jan92 meeting later left and show up
at a small client/server startup responsible for something called a
"commerce server".
we also left and were out doing some consulting. we were brought in to
consult at the small client/server startup because they wanted to do
payment transactions on the server. the startup had also invented some
technology they called "SSL" that they wanted to use; in any case, the
result is now frequently called "electronic commerce".
Part of the "electronic commerce" is something called a "payment
gateway" some past posts
http://www.garlic.com/~lynn/subnetwork.html#gateway
that handles payment transactions from servers on the internet and the
payment network. there had been an "application" first cut ... taking
packets from the internet and reformating them into specification
defined for the payment network. what was missing was a whole lot of
industrial strength stuff that wasn't in the message formats. for
instance, the trouble desk at the part of the payment network had
objective of 5minutes elapsed time to do first-level problem
determination. An early trial of the gateway "application" had a problem
(not working, no transactions) ... and after 3hrs of investigation it
was closed as NTF (no trouble found).
We put together specification for business critical payment gateway
operation ... and the subsequent activity was 5-10 times the activity to
do the original (well developed, well tested) application code. The
result didn't have a significant different total lines of code ... but
there was significant more effort that went into those lines of code.
we also did a JAD with the taligent organization (something of spin-off
of the apple object-oriented "pinK" operating system effort)
... regarding what it would take to turn taligent into basis for doing
business critical data processing ... the net was about 30% change to
their existing libraries and 30% new code (with objective of cutting
development effort for business critical applications by 50-75 percent).
there is also some overlap/similarity between developing code for
business critical applications and developing code for secure
applications (and in some core financial processing applications they
both apply). "secure" development may also include things like
background checks on designers and developers, anybody that is allowed
to touch the code or the operation.
Rob Warnock
+---------------
| rp...@rpw3.org (Rob Warnock) writes:
| > But my "3-5 times" swag [which, by the way, was just a rough estimate
| > based on my recollection of events from ~35 years ago, not any
| > kind of formal academic study] was meant to speak just to initial
| > development/coding costs of code that "just had to work". You are
| > correct that some of that premium could be discounted by lessened
| > maintenance costs, but I wasn't addressing that per se.
|
| I've used "4-10" times to take a "well tested" application and turn it
| into a (industrial/business strength) service.
I won't argue with that number [we're talking rough swags here anyway,
right?], but I *will* claim that there's a *significant* difference
between ``taking a "well tested" application and turning it into a
(industrial/business strength) service'' and writing embedded code
which in its very first field deployment MUST NOT FAIL.
For one thing, once you already have a "well tested application"
it is far too late to apply the techniques I was talking about --
they can really only be done as the code is being initially written.
As the old saw says, "testing can prove the presence of bugs but
never their absence".
For another, the key (IMHO) to *high*-reliability code is to get
more than one set of *competent developer* eyes on it, as it's being
written (not just 100's of QA eyes much, much later). The main way
we did that back at DCA was to make aggressively-critical reviews
of each others' code an integral part of the development process.
That is, suppose you were coding a new bit of protocol code. As soon
as a self-contained section of code [maybe a few definitions, some
global data, a few functions] was ready to compile [yes, *before* the
compiler ever got a look at it!], you'd gather up 2 or 3 co-workers,
book a conference room, print off 3-4 copies of the code, and sit down
together to "defend your thesis" [as it were] to your co-workers:
verbally "prove" that the code worked, while they in turned tried
hard to find a way to break it.
This worked well, but probably only because everyone was a stockholder
[or had options] and because everyone knew that any failure might be
the last for the company. That caused the paranoia to be focussed on
the code itself, and not on one's fellow co-workers. Paradoxically,
this might not have worked as well in a better-funded company! ;-}
It probably also worked well because the project was, by today's
standards, small. The orevious-generation PDP-8 based comm controllers
had had at most 32 KLOC of assembler [much *less* than that, actually,
as much of the 32 KW (12-bit words) of memory was used for buffer space!],
and the new Z-80 based comm controllers we were developing were limited
to 64 KiB [about the same as the PDP-8, roughly], with the boot ROM
being only a fraction of that.
Still, constructing & shipping a new real-time operating system,
buffer-management system, synchronous serial packet-based multiplexing
protocol, and several terminal-specific protocol "shims" with
essentially *no* shipped bugs was... unusual. ;-}
+---------------
| We put together specification for business critical payment gateway
| operation ... and the subsequent activity was 5-10 times the activity to
| do the original (well developed, well tested) application code. The
| result didn't have a significant different total lines of code ... but
| there was significant more effort that went into those lines of code.
|
| we also did a JAD with the taligent organization (something of spin-off
| of the apple object-oriented "pinK" operating system effort)
| ... regarding what it would take to turn taligent into basis for doing
| business critical data processing ... the net was about 30% change to
| their existing libraries and 30% new code (with objective of cutting
| development effort for business critical applications by 50-75 percent).
+---------------
You have my (retrospective) sympathies. It is *VERY* difficult,
if not impossible, to "add quality" to existing code that was
not originally written from Day One to be high-reliability and/or
"bug-free".
Anne & Lynn Wheeler
rp...@rpw3.org (Rob Warnock) writes:
> For one thing, once you already have a "well tested application"
> it is far too late to apply the techniques I was talking about --
> they can really only be done as the code is being initially written.
> As the old saw says, "testing can prove the presence of bugs but
> never their absence".
re:
http://www.garlic.com/~lynn/2009m.html#26 comp.arch has made itself a sitting duck for spam
there were some number that considered that some degree of availability
could be added. one of the interesting issues we had with some some
amount of the payment transactions message formats (that had been
converted to packet/internet operation) ... was that in the original,
there was some amount of impliciit assumption ... that the transaction
messages operated in a circuit enviornment. a straight-forward move of
the transaction message formats, to packet environment, failed to carry
some number of the implicit circuit-based characteristics. part of
retrofitting availability (for "electronic commerce") was compensating
processes for the packet enviornment.
also part of the "retrofit" was failure matrix of 30-40 failure
conditions that might happen in half dozen states ... and being able to
demonstrate automatic recovery ... and/or at least five minute 1st level
problem determination for all cases (not necessarily that the code was
bug free).
it is frequently & significantly more apparent, that attempting to
retrofit security to something (which hasn't been designed from the
groundup), doesn't work.
Duane Rettig
> Kai Harrekilde-Petersen <k...@harrekilde.dk> wrote:
> +---------------
> | > ...US managed company now (...), I can see why: Way too impatient.
> | > Good products take time. If you take the time away, you get nothing.
> | > Not just bad products, no products at all. Business classes often ask why
> | > software is always late and over budget? Answer: Because the schedule is
> | > always too aggressive and the budget grossly understated. Unfortunately,
> | > this answer is only written in Dilbert, not in any MBA program.
> |
> | IMHO, the reason for understating the schedule & cost is simply that
> | if the cost & schedule were correctly state up front, the product
> | would be cancelled by management before it got started.
> +---------------
>
> Indeed. I have some 1970's experience with embedded software that
> simply *had* to be right[1], and I learned two things: (a) It is
> actually possible to develop software that is essentially "correct"
> [bug-free]; and (b) it costs 3-5 *times* as much as "normal" coding
> practice to do so.
> [1] DCA (Atlanta), with the SmartMux 200 series of products.
> The code was in socketed UVPROM, but the company was so
> cash-strapped [and the large UVPROMs we were using at the
> time so expensive] that we literally couldn't *afford* to
> replace all the customers' ROMs to correct a major bug!
> We simply *had* to figure out how not to have any major bugs,
> so we did.
I had similar experiences in a couple of IBM-compatible disk drive
companies in the 70's, as well. Our microcontrollers (all SSI based)
had 40-bit words, and the trick was not to recode whole instructions
(so as to require replacing a whole bank of E-PROMS), but to instead
maniplulate the program in such a way that the fewest E-PROMS had to
be replaced. It wasn't a question of being strapped for money; it was
a question of having such a huge expense for an update, and the trick
was to minimize the size of a field change request.
Of course, the best fix was to have perfect software (ironically,
software which came primarily from the hardware development groups and
was called firmware instead). But the reason such perfection was
approachable was that standards were set, processors were domain-
specific (plus we were competing in the fairly well-defined Plug
Compatible Market), and users didn't try to place new requirements on
product operations (when they did, it resulted in "A New Product
Line" :-)
> There are very few markets which will bear the cost of "bug-free" code.
Or which will stand still long enough...
Duane
Anne & Lynn Wheeler
a relatively trivial "high availabiilty" (but important) thing for the
payment gateway was "multiple A-record" support.
The "payment gateway" was no-single-point-of-failure ... including
multiple links (circuits) into different parts of the internet backbone.
I started out planning on advertising multiple routes ... but in the
process of deployment, the internet backbone announced moving to
hierachical routing only.
this eliminated being able to advertise different routes for the same
ip-address ... and required defining the "name" of the payment gateway
with multiple ip-addresses ("multiple A-record"). webservers that were
contacted the payment gateway then needed to have "multiple-A record"
support. since we had sign-off authority on operation related to payment
gateway ... we could mandate the implementation.
we had several cases we felt that the browser needed to do the same
thing. one of the early adopters of the commerce server (and payments)
was sports product operation that advertised on national sunday football
and expected lots of activity turning half-time. their ISP was operation
that regularly did local presence router maintenance on sundays (they
had schedules where webservers in particular areas wouldn't have service
on particular sundays because their ISP router would be undergoing
maintenance). This was before majority of the internet had any kind of
telco-like provisioning.
So we had meeting with lots of the browser developers where I presented
multiple-A record implementation and the scenarios where it would be
beneficial. The initial response was that it was too complicated and
they weren't going to do it (for the browser/server side ... we could
only advise, we didn't have mandated sign-off authority). I then
provided them with client (telnet, ftp, etc) source code examples from
4.3 Tahoe; no budge. I somewhat hypothesised the issues was that there
were no examples of "multiple A-record" support given in the various
TCP/IP class/text books that they had learned from. It took a year to
get mulitple-A record support into the browser.
as to taligent and "pink" ... there was a period where "object-oriented"
was all the rage, Apple was doing "pink" object-oriented operating
system ... and Sun was doing "spring" object-oriented operating system.
At one point we were invited in and given a run-through of "spring" ...
and then asked if we would consider heading up effort to turn it out as
product (speculation, at least partially, because we had earlier turned
out ha/cmp product).