http://www.onboard.jetbrains.com/is1/articles/04/10/lop/
Above link is Language Oriented Programming...(It might be Beyond
OOP...)
I have gone thru the link ,Its Very good...
Again Can anyone suggest me link for LOP,Groups etc...?
Regards
Raxit Sheth
India
> http://www.onboard.jetbrains.com/is1/articles/04/10/lop/
>
> Above link is Language Oriented Programming...(It might be Beyond
> OOP...)
> I have gone thru the link ,Its Very good...
>
> Again Can anyone suggest me link for LOP,Groups etc...?
Bible, the Tower of Babel? (:-))
--
Regards,
Dmitry A. Kazakov
http://www.dmitry-kazakov.de
Plz Dont Tell any msg Ambiguously....
What is the meaning of your reply..., I cant Understand...?
So plz ,re-reply..
Thanks
Regards
Raxit Sheth
The danger of trying to reinterpret Mr Kazakovs response is dangerous,
but I don't think he's was fan.
My reading of it was the "the emporer has no clothes", i.e. I could not
see anything in the article which was not mainstream OO, and IDE
technology.
Defining problem domain specific language is what problem domain
specific OO class libraries are about (and procedural ones), IDE's that
interpret the 'structure' of our 'language' is what my IDE does now, I
type an int variable name and put a '.' after it and it examins the
metadata and tells me I can invoke ToString, add, subtract etc.
I only gave it a quick once through though, so I may have missed the
point.
> What is the meaning of your reply.
That I can't take LOP seriously.
> Hello All...
>
> http://www.onboard.jetbrains.com/is1/articles/04/10/lop/
There are two directions one can go with this. One is to create DSLs at
the 3GL level. The other is to create a language that allows one to
abstract arbitrary domains directly in domain terms.
There is no question that 3GL DSLs can improve developer productivity
and software reliability compared to traditional 3GLs. The downside is
that they represent a high degree of specialization; the developer must
learn a new DSL to solve a problem in a new domain. This can be a
problem for larger applications because they often have subject matters
from different domains and sometimes those concerns overlap. For
example, an Inventory Control application needs a knowledge of the goods
being inventoried and associated business processes. But it also
requires a knowledge of operations research forecasting algorithms.
Another problem is dealing with the computing space. Almost all
commercial software critically involves mapping a business domain into
the domain of hardware computational models. One must necessarily
manipulate the business abstractions in a manner that provides
optimization in terms of the hardware models. To avoid explicitly
dealing with that optimization the DSL compiler must provide
optimization for the computing space. But the only way that can be done
is if the DSL itself is sufficiently abstract so that it is independent
of local computing environments. However, that segues to the second
evolutionary road.
If one has sufficient abstraction so that the solution representation is
independent of the local computing environment, one has a 4GL. To
provide that level of independence one needs a high degree of
abstraction. In particular, one needs to be able to abstract business
concepts directly in those terms simply because one can't use computing
space terms. That happens to be exactly what OOA does. If one has a
general purpose language capable of such abstraction, one doesn't need a
DSL because one can directly abstract exactly the same way the DSL does.
UML represents such a 4GL and translation technology allows transparent
optimization for the local computing space through the transformation
engine. Translation evolved as an extension of optimization for the
deterministic (albeit highly complex) computing space. But it ended up
in the same place where 3GL DSLs are destined to end up: a general
purpose language for directly abstracting the problem space purely in
problem space terms and then compiling executables from it that are
optimized for the computing environment.
Bottom line: I agree with Kazakov that there is nothing in LOP that is
not already addressed in OOA when combined with MDA transformation.
IOW, it is too little, too late.
*************
There is nothing wrong with me that could
not be cured by a capful of Drano.
H. S. Lahman
h...@pathfindermda.com
Pathfinder Solutions -- Put MDA to Work
http://www.pathfindermda.com
blog: http://pathfinderpeople.blogs.com/hslahman
(888)OOA-PATH
Speaking of Language Oriented Programming, how does English sound?
Your Voice Might be true,,...
But what I am trying to put here in in the First Section ,
Author/Editor of LOP has put Some Good Thing that one Should not BOUND
To the Language. And He was feeling that He is Bounded by the
Constraint of X,Y,Z ...Language...That i think is Real....
again the solution describe might not be good, But one more point i
think is important..."Instead of Writing TEXT CODE, In future...its
GRAPHICAL( or any Non-Text) Entity".
> There is no question that 3GL DSLs can improve developer productivity
> and software reliability compared to traditional 3GLs. The downside is
> that they represent a high degree of specialization; the developer must
> learn a new DSL to solve a problem in a new domain. This can be a
> problem for larger applications because they often have subject matters
> from different domains and sometimes those concerns overlap. For
> example, an Inventory Control application needs a knowledge of the goods
> being inventoried and associated business processes. But it also
> requires a knowledge of operations research forecasting algorithms.
Third Point is If XML(General Purpose Lang) Is there then why there is
need to specify more (Specific) Lang like SAML,VXML,CCXML,...?
I think I disagree with this somewhat, as I have my own, my very own,
theory (think the old Monty Python routine) that computation is more a
unipolar constructive enterprise than a mapping, but that's another
rant. The use of the term "optimization" sort of jumped out at me
here, aren't we satisficed if it at least works, isn't optimization
sort of a secondary issue?
>If one has sufficient abstraction so that the solution representation is
>independent of the local computing environment, one has a 4GL. To
>provide that level of independence one needs a high degree of
>abstraction. In particular, one needs to be able to abstract business
>concepts directly in those terms simply because one can't use computing
>space terms. That happens to be exactly what OOA does. If one has a
>general purpose language capable of such abstraction, one doesn't need a
>DSL because one can directly abstract exactly the same way the DSL does.
You mean, that's what OOA *tries* to do.
>Bottom line: I agree with Kazakov that there is nothing in LOP that is
>not already addressed in OOA when combined with MDA transformation.
>IOW, it is too little, too late.
I agree, too.
J.
> that computation is more a
> unipolar constructive enterprise than a mapping, but that's another
> rant.
Can you explain, it sounds interesting and it may be a very interesting
rant.
> But what I am trying to put here in in the First Section ,
> Author/Editor of LOP has put Some Good Thing that one Should not BOUND
> To the Language. And He was feeling that He is Bounded by the
> Constraint of X,Y,Z ...Language...That i think is Real....
The problem is that DSLs are 3GLS (maybe 3.nGLs where n -> 0) and I
don't think that is where the future is. When the DSL gets abstract
enough to be independent of the hardware computational models, one will
be at the 4GL level where one creates the problem space abstractions
directly anyway.
The bottom line is that I already use a 4GL whose syntax allows me to
abstract at the same level of abstraction as the DSL for any problem
domain. For example, a DSL for accounting would have a built-in
abstraction for T-Account. But if I am doing a general ledger
application via OOA abstraction, I am going to have a T-Account class
too. I just abstracted directly it from the problem space using the
same domain knowledge that someone using the DSL would have to have to
use the DSL properly. The difference is that I only have to learn one
notation syntax for any problem space because my notation is
sufficiently general to provide abstractions like T-Account naturally on
an as-needed basis.
>
> again the solution describe might not be good, But one more point i
> think is important..."Instead of Writing TEXT CODE, In future...its
> GRAPHICAL( or any Non-Text) Entity".
Note that 4GLs like UML or HPVEE /are/ predominantly graphical.
However, there are contexts where text is more appropriate.
For example, when I started with translation in the late '80s one
described what went on within methods using Action Data Flow Diagrams.
Today, though, all the translation tools use a text-based abstract
action language. That's because one can express anything in an ADFD in
text much more compactly and the text is a lot easier to edit. [When a
process needed to be added it was always in the middle of the densest
part of the diagram and one spent ten minutes rearranging bubbles to
make the diagram readable. But in AAL one just inserts a single line.]
IOW, the market made a decision that text was better in that situation.
The reason that compaction ratio applies is because of the nature of
methods descriptions. Methods essentially implement process algorithms
so all one needs to define is sequence, iteration, and branching. The
processes are always abstracted as fundamental units that are
manipulated in the sequence, iteration, and branching. Mathematicians
have been providing efficient text notations for that sort of symbolic
manipulation for millennia.
>>There is no question that 3GL DSLs can improve developer productivity
>>and software reliability compared to traditional 3GLs. The downside is
>>that they represent a high degree of specialization; the developer must
>>learn a new DSL to solve a problem in a new domain. This can be a
>>problem for larger applications because they often have subject matters
>>from different domains and sometimes those concerns overlap. For
>>example, an Inventory Control application needs a knowledge of the goods
>>being inventoried and associated business processes. But it also
>>requires a knowledge of operations research forecasting algorithms.
>
>
>
> Third Point is If XML(General Purpose Lang) Is there then why there is
> need to specify more (Specific) Lang like SAML,VXML,CCXML,...?
I would say No. FWIW, I think XML is too 3GLish to be useful for
general customer problem space abstraction.
>>Another problem is dealing with the computing space. Almost all
>>commercial software critically involves mapping a business domain into
>>the domain of hardware computational models. One must necessarily
>>manipulate the business abstractions in a manner that provides
>>optimization in terms of the hardware models. To avoid explicitly
>>dealing with that optimization the DSL compiler must provide
>>optimization for the computing space. But the only way that can be done
>>is if the DSL itself is sufficiently abstract so that it is independent
>>of local computing environments. However, that segues to the second
>>evolutionary road.
>
>
> I think I disagree with this somewhat, as I have my own, my very own,
> theory (think the old Monty Python routine) that computation is more a
> unipolar constructive enterprise than a mapping, but that's another
> rant. The use of the term "optimization" sort of jumped out at me
> here, aren't we satisficed if it at least works, isn't optimization
> sort of a secondary issue?
It depends on the subject matter. I spent two decades in R-T/E working
on megabuck machines where the hardware was an order of magnitude faster
than the computer feeding it and the customer wants throughput for that
investment. I spent another decade solving np-Complete problems on
machines that were too small. So from my perspective performance is
often as important as correctness in practice.
<hot button>
Years ago Djikstra wrote a piece for one of the professional rags
bewailing code bloat and downgrading performance. (Anyone remember when
one could run a spreadsheet on an Apple II, whose cycle rate was
measured in KHz, with 128 Kb of memory and a 700 Kb floppy disk?) At
the time I thought he was, once again, ahead of his time. Moore's Law
has lulled a lot of the industry into believing that performance
problems can be solved by waiting for next month's bigger and better
machine.
IT has been particularly affected because disk access time has not
dropped as quickly as CPU cycles/sec have grown so the traditional I/O
bottleneck has just gotten worse and software optimization has become an
arcane issue for server engines. Got a performance problem? Buy
another server.
The result is that the industry has largely forgotten how to optimize so
I think there is going to be a nasty surprise coming up in the next
decade. Moore himself has expressed concern that his Law is likely to
fail soon. But far more important, IMO, is the fact that the problems
are beginning to catch up to the machines. That is, until recently the
size of the problems being solved has not grown as quickly as computing
power, especially in IT where the thing that tends to scale is I/O volume.
One reason is that until relatively recently we simply didn't have the
productivity and quality control to attack truly large problems in
commercial environments. Really big problems, like atmospheric
diffusion, were solved on mainframes by very bright post-graduate slave
labor who didn't have to worry a lot about maintainability because the
requirements (algorithms) didn't change much.
Today, though, we have the technology, tools, methodologies, and project
management processes that enable small teams to produce MLOC
applications with fairly reasonable quality in a fairly short time. So
the /business/ problems that /can/ be solved on desktops are starting to
increase in size and I see that growth as approaching exponential until
it catches up to machine power -- simply because of the productivity and
QA gains among developers. IOW, since we have more capacity for problem
solving, we will solve bigger problems.
We already see this happening for OSes and office tool suites. Today as
much as 60% of the CPU cycles is devoted to activities (GUI, OS
housekeeping, interoperability) peripheral to the user's problem and the
fraction is growing despite Moore's Law. Stuff more and bigger and more
bloated applications into that environment and we are going to have a
problem even with Moore's Law still working. One manifestation is that
we have to keep upgrading our desktops as we upgrade software even
though we may still be doing pretty much the same things.
Perhaps one of the best examples is interoperability. Today we have
lots of infrastructures for enhancing developer productivity,
particularly for interoperability. Things like remote object access
make life a whole lot easier for the software developer. The problem is
that the infrastructure is not free; it carries horrendous overhead
compared to simple data transfer messages. But the developer would have
to worry about protocols and encode/decode details on either end without
the infrastructure. The result is awful performance that contributes to
the huge overhead in CPU cycles.
That's a trade-off between short-term developer cost vs. long-term cost
in the execution environment. The vendor always opts for developer
productivity because that reduces out-of-pocket expenses and the
long-term cost is borne by the customer in upgrading hardware. So far
the industry has gotten away with this because of hardware costs
dropping like a stone. But PCs are now commodities, the CPU is only a
small part of the cost of a PC, and there is no Moore's Law for the
rest. So we are approaching a stabilization in hardware pricing.
When commodity prices stabilize to close to cost-of-living increases and
the problems increase in size, then the cost of software performance
will become more apparent to the customers through the cost of
upgrading. At that point the industry is in for a rude awakening.
[Actually, I expect this wakening to be awhile mainly because there is
another rude awakening that is more imminent. Customers are beginning
to realize that the only things that break in their lives nowadays are
software. Very soon reliability is going to become a major competitive
advantage in software development, just as it became in the '80s for
"hard" industry. (One example is the market share hit WinX is taking
over Linux reliability.) Unfortunately moving commercial software from
4-Sigma to 5-Sigma and beyond is going to take awhile so the winners are
going to be those who got on board earliest. But I digress...]
</hot button>
>
>
>>If one has sufficient abstraction so that the solution representation is
>>independent of the local computing environment, one has a 4GL. To
>>provide that level of independence one needs a high degree of
>>abstraction. In particular, one needs to be able to abstract business
>>concepts directly in those terms simply because one can't use computing
>>space terms. That happens to be exactly what OOA does. If one has a
>>general purpose language capable of such abstraction, one doesn't need a
>>DSL because one can directly abstract exactly the same way the DSL does.
>
>
> You mean, that's what OOA *tries* to do.
Remember that you are talking to a translationist. B-) Because our OOA
model is eaten by a code generator we are forced to apply the
methodological rigor necessary to ensure separation of customer and
computing concerns. IOW, we /always/ do what elaboration OOA modelers
/should/ do but often don't.
Implementation pollution will get translation reviewers to bring out the
crucifixes and garlic cloves faster than anything else. The acid test
for an OOA model is whether it could be unambiguously implemented as a
manual system in the customer's environment without change (however
inefficient that might be).
> Speaking of Language Oriented Programming, how does English sound?
Sounds like Cobol.
--
mail1dotstofanetdotdk
It does.
> But the developer would have
> to worry about protocols and encode/decode details on either end without
> the infrastructure.
Instead they end up worrying about fine tuning the
serialization/deserialization process, for example, overriding
writeObject/readObject in Java to null out some derived data before
sending and looking it up on the other side from a local cache.
> The result is awful performance that contributes to
> the huge overhead in CPU cycles.
>
It is awful. The bottleneck isn't generally the CPU, just the sheer
amount of stuff that is going over the wire.
System administrators end up worring about the size of the pipe to
remote locations, and often end up using Citrix.
Regards,
Daniel Parker
Well, I hear you, but even so.
><hot button>
>Years ago Djikstra wrote a piece for one of the professional rags
>bewailing code bloat and downgrading performance. (Anyone remember when
>one could run a spreadsheet on an Apple II, whose cycle rate was
>measured in KHz, with 128 Kb of memory and a 700 Kb floppy disk?) At
>the time I thought he was, once again, ahead of his time. Moore's Law
>has lulled a lot of the industry into believing that performance
>problems can be solved by waiting for next month's bigger and better
>machine.
Microsoft has been a leading offender here. The year I graduated
college, IBM mainframe (core, 2 microsecond) memory was a nice, round
one million dollars (that's about $4m in current dollars) per
megabyte, for a 2 MIP 370/168 that might support a hundred TSO and CMS
users on 2741 Selectric and 327x green (or color!) channel-connected
CRTs that cost about 2x what PCs cost. Girl friend employed there was
assigned to enhance IMS performance running in an average sized 256KB
(kilobyte) partition. Ah yes, those were the days!
>IT has been particularly affected because disk access time has not
>dropped as quickly as CPU cycles/sec have grown so the traditional I/O
>bottleneck has just gotten worse and software optimization has become an
>arcane issue for server engines. Got a performance problem? Buy
>another server.
Well, yes and no. Someone always wants more, and the numbers are as
you say, but now typical Wintel networks have terabyte SAN/NAS (RAID)
boxes on gigabit channels, which helps overall thruput, it's not news
by mainframe standards but it's pretty darned fast, pilgrim, long as
you don't mind at least a little latency. And the processors are just
ridiculously fast, even though Wintel still doesn't grok large degrees
of parallelism and has various difficulties loading (dirt-cheap!) RAM
about 4gb or so. So yes, the gap between processor and disk may grow,
but the overall power available for commercial apps is huge. What I
mostly see around town is endlessly sloppy code that gets by because
the hardware is sufficiently fast.
>The result is that the industry has largely forgotten how to optimize so
>I think there is going to be a nasty surprise coming up in the next
>decade. Moore himself has expressed concern that his Law is likely to
>fail soon. But far more important, IMO, is the fact that the problems
>are beginning to catch up to the machines. That is, until recently the
>size of the problems being solved has not grown as quickly as computing
>power, especially in IT where the thing that tends to scale is I/O volume.
I've been doing a lot of database the last few years, and there is a
common recruiting ad, "must have experience with databases of at least
300gb". Huh? What they're really after is, yes, someone who has some
sense of scalability and bandwidth performance tradeoffs. Well, it is
to laugh. I learned my performance tuning on systems three orders of
magnitude slower, yes, I recall working on a 200mb database, trying to
keep it from bogging on the superminicomputer it ran on. The several
times I've worked with 300gb database performance, did I do all sorts
of numeric analysis of the server and exotic nth-degree normalizations
and denormalizations? Nah. I run a few logs, identify the
bottlenecks, universally find a missing index, ghastly code, or an
optimizer bug, and by the time I've finished the easy stuff I've
increased performance by 300% or so and everyone loses interest in
doing anything *interesting*! Generally I arrive just after they've
installed a new server which has *already* decapitated their load
peaks, and between that and my prelims, I soon move on to
non-performance issues.
So, in the mundane commercial world, I don't see performance as much
of an issue in the immediate future (although doing the simple-minded
stuff is turning into my own bread and butter!). Microsoft remains an
immense resource-pig, capable of putting wait loops and malformed
concurrency models deep in their engines, so faster processors get in
their own way at ever-higher speeds, but even Microsoft in Server2003
and COM 1.5 is finally getting the platform straightened out, about
where OS/360 was in 1968, maybe.
Comments on Wintel hyperthreading, anyone?
Josh
Is it possible in theory to create a generate-and-test system that
will converge on a useful system, or must software be deduced in a
rational, semantic process from real-world models?
The generate-and-test question I see as that unipolar constructive
thing, even though the -test comprises a feedback loop so it's not all
that unipolar, I guess. Since lots of people have spent their time on
genetic and evolutionary programming models, apparently there are some
out there who think this might work. But look just how different that
is from the more common, conventional, analytic, similarity model.
Generate-and-test sits at the opposite end of the spectrum from MDA,
and it's hard to envision a universe in which both approaches are
truly valid. I want to do some compare and contrast, and some really
deep diving for fundamentals, to resolve the paradox.
As an exercise for the reader, position your favorite methodology on
this spectrum.
That's about the nickel tour.
Thanks for asking!
Josh
>><hot button>
>>Years ago Djikstra wrote a piece for one of the professional rags
>>bewailing code bloat and downgrading performance. (Anyone remember when
>>one could run a spreadsheet on an Apple II, whose cycle rate was
>>measured in KHz, with 128 Kb of memory and a 700 Kb floppy disk?) At
>>the time I thought he was, once again, ahead of his time. Moore's Law
>>has lulled a lot of the industry into believing that performance
>>problems can be solved by waiting for next month's bigger and better
>>machine.
>
>
> Microsoft has been a leading offender here. The year I graduated
> college, IBM mainframe (core, 2 microsecond) memory was a nice, round
> one million dollars (that's about $4m in current dollars) per
> megabyte, for a 2 MIP 370/168 that might support a hundred TSO and CMS
> users on 2741 Selectric and 327x green (or color!) channel-connected
> CRTs that cost about 2x what PCs cost. Girl friend employed there was
> assigned to enhance IMS performance running in an average sized 256KB
> (kilobyte) partition. Ah yes, those were the days!
I guess I have a few years on you. B-) My fond memories are of plug
boards when changing vacuum tubes was part of a Programmer's job
description. I also have fond memories of ka-chunk, ka-chunk, ka-chunk
-- using teletype machines with 5-lb piston keys. Those were such
character-building experiences that I quit software for a decade until
ca '68 when there was high tech stuff like your 3270 terminals (always
green then).
>>IT has been particularly affected because disk access time has not
>>dropped as quickly as CPU cycles/sec have grown so the traditional I/O
>>bottleneck has just gotten worse and software optimization has become an
>>arcane issue for server engines. Got a performance problem? Buy
>>another server.
>
>
> Well, yes and no. Someone always wants more, and the numbers are as
> you say, but now typical Wintel networks have terabyte SAN/NAS (RAID)
> boxes on gigabit channels, which helps overall thruput, it's not news
> by mainframe standards but it's pretty darned fast, pilgrim, long as
> you don't mind at least a little latency. And the processors are just
> ridiculously fast, even though Wintel still doesn't grok large degrees
> of parallelism and has various difficulties loading (dirt-cheap!) RAM
> about 4gb or so. So yes, the gap between processor and disk may grow,
> but the overall power available for commercial apps is huge. What I
> mostly see around town is endlessly sloppy code that gets by because
> the hardware is sufficiently fast.
Right. Any improvement in I/O throughput via parallelism, caching, or
whatever will just make it less easy to get away with pig code in
middleware or on the client side. Similarly, improvements at the
middleware level will also push back on client performance. That is,
client screwups just become more noticeable.
I suspect the problems you end up fixing to get the first 300% are often
the result of developers simply not caring about performance because of
that obvious hardware gap. IOW, they look at the hardware gap and
naively assume hardware will take care of everything.
I'd say the most common causes are:
* Some maintenance operation dropped the primary key and nobody
noticed.
* Someone felt the table didn't deserve a primary key, which might
even have been true at some time zero for all I know, though in my
world everything has a real PK plus or minus an identity/surrogate.
* They make some code change and never even bother to check
performance, which may (if checked) even seem OK especially in test
environments but may degrade exponentially over time on production
volumes.
--
IOW, *really* dumb stuff.
J.
>>I suspect the problems you end up fixing to get the first 300% are often
>>the result of developers simply not caring about performance because of
>>that obvious hardware gap. IOW, they look at the hardware gap and
>>naively assume hardware will take care of everything.
>
>
> I'd say the most common causes are:
>
> * Some maintenance operation dropped the primary key and nobody
> noticed.
>
> * Someone felt the table didn't deserve a primary key, which might
> even have been true at some time zero for all I know, though in my
> world everything has a real PK plus or minus an identity/surrogate.
My world too. [On those rare occasions where I have to deal with an
RDB. B-)] But both these reflect more on fundamental competence;
they're the sort of thing a Programming Manager should be breaking
people's thumbs over.
>
> * They make some code change and never even bother to check
> performance, which may (if checked) even seem OK especially in test
> environments but may degrade exponentially over time on production
> volumes.
My suspicion is that they never bother to check because Conventional
Wisdom tells them that performance isn't an issue anymore.
Ah, you said the magic words that get me going, "Programming Manager".
The shop I was just in, several hundred people in the IT organization,
and to a first approximation there is nobody there who is even
supposed to be a "programming manager". There are "project managers"
who run Gantt charts and go to meetings, but they are not expected to
know anything about software development, though I suppose they're
supposed to be able to say "life cycle" with a straight face. There
are "project leads" who are supposed to be expert in eleven different
products on three different tiers, but they are not really expected to
care about, know about, or create design documents or other non-coding
project artifacts. All decisions are made by the end-users or their
managers or their SMEs - none of whom, we will note, is even supposed
to know anything about IT. The head of the database admin and
development group, about twenty people by itself, spouted some
ridiculous database guidelines that Microsoft has been spewing around
the industry the past few years and apparently nothing more, and his
functional focus, again, did not really include anything one would
call "programming management". It seems a system carefully devised to
AVOID putting any responsibility on anyone in the IT system. I gather
things are different where you are, but I've seen this kind of thing
in many shops these days, one will note that it goes along
hand-in-glove with the idea of outsourcing work 10,000 miles away, or
bringing in minimally qualified developers of any sort, since there is
no call for them having to actually know anything about the business,
the application, or software development processes or methodology. It
seems the only filter on hiring is the resume parsing program used by
HR (that's your older brother, right? :)), and that's a whole
additional rant.
Note that I've barely touched on the issue of "fundamental
competence", though I do a bit more below.
And, oh, how well does it work? Do you need to ask?
>> * They make some code change and never even bother to check
>> performance, which may (if checked) even seem OK especially in test
>> environments but may degrade exponentially over time on production
>> volumes.
>
>My suspicion is that they never bother to check because Conventional
>Wisdom tells them that performance isn't an issue anymore.
Well, to a large degree, it *isn't* an issue anymore, that's exactly
what I've been saying, because the servers are just too, too fast!
I'm talking major websites by major corporations here, with gawd-awful
sloppy code. In one case before I did my 300% deal, they had just
spent the previous year with a team of five hand-tuning stuff to keep
the load peaks within the server capacity. Yeah, it was an issue when
it became an issue, but rather blatently, that team of five that
preceeded me was far from thorough. The first thing they do when it
becomes an organizational issue is to upgrade the server. That's not
unreasonable, I suppose, the $30k or so it might cost, is just a few
month's pay for a competent developer, and see above for just how
competent that is likely to be. So, I think I'd suggest that it is an
issue, but most developers today (in reality, not in HR fantasies)
actually know only one or two specialized areas (and are seriously
proud of their "focus"), so the average developer in practice does not
spend much time or real expertise on performance, heck, it's too much
a systems-level integrated issue to even be on the conceptual radar
screen of the kinds of developers they hire in most commercial IT
shops. Who, as you said in one thread or another, are done with the
code in any case as soon as it compiles, so just exactly when were
they supposed to measure the performance anyway?
</rant>
Josh
> And, oh, how well does it work? Do you need to ask?
That's why most shops spend so much time looking for Life Preservers.
You're right, most shops aren't a lot of fun. So when one finds a good
one, one tends to stick. When I retired after 21 years I was the only
one in the shop who had ever worked anywhere else. (Prior to that I had
never worked more than three years anywhere.) When I left there were
people still there who had more time in than I and the average tenure
was probably 8-10 years. [Virtually all the attribution was the more
ambitious types moving on to Management elsewhere in the company; one
Division Manager, two Division Engineering Managers, one Division
Hardware Manager, and a gaggle of software functional managers. (We
were a bit top heavy with superstars.)]
How much money would they save if they didn't need to upgrade the
servers so often? The alternative may not be feasible because of the
low competence level. But there is no optimization competence because
of the Conventional Wisdom. Catch-22.
However, if the competence level were raised, that optimization
expertise could be spread over a lot more than servers. Once the basic
competence is there, the marginal cost of exercising it is trivial.
>Hello All...
>
>http://www.onboard.jetbrains.com/is1/articles/04/10/lop/
>
>Above link is Language Oriented Programming...(It might be Beyond
>OOP...)
>I have gone thru the link ,Its Very good...
I don't know what to think of this yet. On the one hand it sounds a
bit too ambitious. On the other hand, there is no doubt that intelliJ
is one of the best (if not THE best) Java IDE out there. It took a
lot of skill and creativity to conceive and build it. So when the
creator of IntelliJ speaks, I'm going give what he says due
consideration.
-----
Robert C. Martin (Uncle Bob) | email: uncl...@objectmentor.com
Object Mentor Inc. | blog: www.butunclebob.com
The Agile Transition Experts | web: www.objectmentor.com
800-338-6716
"The aim of science is not to open the door to infinite wisdom,
but to set a limit to infinite error."
-- Bertolt Brecht, Life of Galileo
"I'm talking about the limitations of programming which force the
programmer to think like the computer rather than having the computer
think more like the programmer."
Is there some advantage to thinking like a computer? Most of the
people I know that program are far better problems solvers than those
that don't program. Does thinking like a computer help break down a
problem and see sides of it that we didn't see when we first were
introduced to the problem?
"To understand what Language Oriented Programming is, let's first
take a look at today's mainstream programming. It goes something like
this:
Think:You have a task to program, so you form a conceptual model in
your head about how to solve the problem.
Choose: You choose some general-purpose language (such as Java or C++)
for writing the solution.
Program: You write the solution by performing a difficult mapping of
your conceptual model into the programming language."
Is this process actually a cycle instead of a linear process? Are they
discrete steps or do they in fact overlap? What exactly is going on in
the programming step? Why is it taking so long? Is it because the
programmer is trying to figure out how to translate the code or is
there a good deal of thinking and exploration of the problem space
taking place in this step?
"Now, the Program step is much less of a bottleneck because the DSLs
make it much easier to translate the problem into something the
computer can understand"
This is only true if the only action that takes place during the
programming step is translation of the problem into code.
"For me, the most serious problem is that there is a very long gap
between when I know exactly how to solve a problem and when I have
successfully communicated this solution to the computer as a
program."
Is this the bottle neck of the majority of programmers? Or is it only a
problem when implementing a problem type that have been solved many
time in a similar environment? i.e. seasoned programmers.
"For example, today a good deal of development time is spent on
object-oriented design (OOD). This is actually a fairly creative
process where the programmer expresses classes, hierarchies,
relationships, and such. ... The process of OOD is necessary because
these classes and methods are the only abstractions that
object-oriented languages understand. It seems like it is necessary and
creative, but with Language Oriented Programming, OOD is not needed at
all."
My understanding (correct me if I'm in error) is that OOP is about
making classes and methods that the object oriented languages
understand but OOD is about applying a design technique to better
understand the problem space. Specifically the objects and how they
interact. Why is this no longer needed in LOR? Does LOR provide a
different technique for analyzing the problem?
"We were taught that computers are modeled after the Turing machine,
and so they 'think' in terms of sets of instructions. But this view
of programming is flawed. It confuses the means of programming with the
goal."
I would be interested in a more detailed explanation of how the Turing
machine model "confuses the means of programming with the goal" I
thought the Turing machine model allowed us to determine the
solvability of a problem and the required steps to solve that problem.
The fact that it was easily programmable, I thought, was a useful side
effect of this mathematical representation of problems that was
developed before modern computers.
When I have a problem to solve, I think of the solution in my head.
This solution is represented in words, notions, concepts, thoughts, or
whatever you want to call them.
What about problems where the solution is not immediately evident? Or
very large problems like implementing an ERP system where the solution
has a lot of complexity deriving, not from difficult isolated problems
but from the interaction of many simple problems?
"This is the main reason I think programmers should have the freedom
to create their own languages-so they can express solutions in more
natural forms."
I frequently will gain insight into a problem by resolving it in
another language or expressing it in a different form that the one that
is "natural".
> "For me, the most serious problem is that there is a very long gap
> between when I know exactly how to solve a problem and when I have
> successfully communicated this solution to the computer as a
> program."
This seems a little bit of a contradiction. If the guy *knows exactly*
how to solve the problem, what's so hard about coding the solution?
After all, it's generally accepted that coding is not the hardest part
in program development...
Interesting thoughts. From the paper cited above:
-------
"For me, the most serious problem is that there is a very long gap
between when I know exactly how to solve a problem and when I have
successfully communicated this solution to the computer as a program. I
can explain the problem and solution to another programmer in a matter
of hours, but encoding this solution into the computer takes much
longer. This is because with a programmer I can use natural language
which is very rich, but for the computer, I must use a general-purpose
programming language which is much less expressive."
-------
Programmer make huge assumptions about the details. When explaining
something to another programmer we don't have to be specific. We can be
ambiguous, and that's the problem with "natural" languages. They're too
ambiguous. Programs must do specific things, and they require specific,
unambiguous instructions to follow.
It takes time just to discover the details before coding, or the lack of
details after the coding.
The paper goes on to say:
-------
"The process of OOD is necessary because these classes and methods are
the only abstractions that object-oriented languages understand. It
seems like it is necessary and creative, but with Language Oriented
Programming, OOD is not needed at all."
------
What are we supposed to imagine here? Each of us with our own
perspective (favorite language) might either agree or disagree. I'm
curious from what frame of reference the author writes from. When the
paper throws around OO are they shortcomings found in specific OO
languages? Java and C++ drive me nuts and I'd tend to agree with any OO
criticism if it were based on either. However, not all OO languages are
similarly restricted, just as not all programmers are similarly
restricted. Some languages already facilitate the creation of DSL-like
constructs by leaving /everything/ open to extension and tailoring by
programmers. Nothing is final. Nothing is protected.
More:
------
"The traditional way to address this problem is to write comments or
other forms of documentation to capture the design and model
information. This has proven to be quite a weak solution for a number of
reasons, not the least of which is the cost of writing such auxiliary
documentation, and the tendency of documentation to grow out-of-synch
with code."
This problem is not unique to programming. Our company is experiencing
a growth spurt and investors are requiring written policies and
procedures. We were amazed at how many "procedures" were being
performed informally. How did we know how to do that? How were so many
important functions being performed if there weren't instructions?
After documenting them we performed an audit a few months later. The
procedures had changed again even though they were "documented". In
some cases they're barely recognizable.
The lesson to get from this is HUMANS are incredibly flexible and
adaptable, and can respond in real-time to change. HUMAN procedures
don't generally go through QA and aren't system integrated. If a human
tries something and it throws an exception, they fix it there-and-then
and proceed. Nearly ALL companies would throw conniptions if
programmers debugged a running application, fixed the code, and allowed
the program to continue from there.
The next paradigm in programming should be modeled after how humans deal
with procedure and exceptions, and learn from it. There has already
been a lot of work done in this area and I've ready some interesting
results in books on complexity and artificial life using real-world
practical examples, like a system that "watched" a real human route gas
around the country including problems, broken lines, strikes, etc.
After some time (I can't remember how much) the system was allowed to
take over and did a remarkable job -- even finding ways around problems.
Besides its ability to "learn" another important feature was positive
and negative feedback. You don't know if what you did was good if there
isn't some reinforcement.
Anyway, interesting article. Some great insight.
> The next paradigm in programming should be modeled after how humans deal
> with procedure and exceptions, and learn from it.
Definitely a profound idea. I'm far from sure that we're anywhere near
ready for that - we're encumbered by ideas about cognition and computing
that make it difficult to even think of stating the problem that way.
> Our company is experiencing a growth spurt and investors are
> requiring written policies and procedures. We were amazed at
> how many "procedures" were being performed informally. How
> did we know how to do that?
I'd suggest asking another question - *why* were you so surprised ? What
were you expecting ? :)
> The lesson to get from this is HUMANS are incredibly flexible and
> adaptable, and can respond in real-time to change.
Not just humans - so do many other systems geared to survival, or at
least stability. Stable systems show equifinality - they get to the same
end state irrespective of starting conditions (or perturbations) within
some range of tolerance. Species in an ecosystem are intricately
interdependent, but most ecosystems can afford to lose one species.
The software systems we're able to design also show interdependence, but
when one module fails (even a small one) typically the whole thing
fails. The famous Ariane crash is a good example of this fragility; the
bug that was the ultimate cause of the crash had caused a fault that was
entirely irrelevant, at the time it occurred, to keeping the rocket in
the air. A more proximate cause of the crash was that the system
interpreted an error message as if it had been a command to the
boosters.
Think about it. That's a bit like a car driver hearing someone sneeze...
But rather than disregard the "utterance" as irrelevant, the driver is
compelled to assing meaning to it. Arbitrarily, the driver decides it
means "turn right all the way". As errors go, it's totally bizarre. And
yet so many systems "designed" under the prevailing assumptions as to
what constitutes "design" exhibit this kind of behaviour, this mix of
awesome intelligence and utter stupidity. (I'm reminded of the quality
Doug Hofstadter calls "Sphexishness".)
Indeed.
> > The lesson to get from this is HUMANS are incredibly flexible and
> > adaptable, and can respond in real-time to change.
Without being incredibly flexible and adaptable and responding in
real-time to change - I wouldn't last a minute driving on US101 ;-)
> Not just humans - so do many other systems geared to survival, or at
> least stability. Stable systems show equifinality - they get to the same
> end state irrespective of starting conditions (or perturbations) within
> some range of tolerance. Species in an ecosystem are intricately
> interdependent, but most ecosystems can afford to lose one species.
>
> The software systems we're able to design also show interdependence, but
> when one module fails (even a small one) typically the whole thing
> fails. The famous Ariane crash is a good example of this fragility; the
> bug that was the ultimate cause of the crash had caused a fault that was
> entirely irrelevant, at the time it occurred, to keeping the rocket in
> the air. A more proximate cause of the crash was that the system
> interpreted an error message as if it had been a command to the
> boosters.
Seems that you are picking and choosing the cases to suit your
conclusion. Ecosystems can afford to lose one species - except when
they can't and the whole thing unravels (aka fragile ecosystem). When
one module fails (even a small one) typically the whole thing continues
on as though nothing happened (aka log the error).
Ariane is a reminder that "accidents usually have many necessary causal
factors, and if you eliminate any single one of them, the accident
would not have happened".
http://sunnyday.mit.edu/accidents/Ariane5accidentreport.html
http://www.rvs.uni-bielefeld.de/publications/Reports/ariane.html
> Think about it. That's a bit like a car driver hearing someone sneeze...
> But rather than disregard the "utterance" as irrelevant, the driver is
> compelled to assing meaning to it.
This silly example isn't anything like the Arianne accident, but let's
make it slightly less silly - That's a bit like a car driver hearing a
front passenger shriek "Oh no!", distractedly turning to look at the
passenger, and crashing into the slowing traffic in front of them.
There are situations in which the "Oh no!" may be relevant, but they
don't justify looking away from the road.
> Ecosystems can afford to lose one species - except when
> they can't and the whole thing unravels (aka fragile ecosystem).
Do we know of ecosystems that collapsed for loss of a single species ?
More generally, do we know of many ecosystems that are fragile, other
than the ones whose fragility we are responsible for ? (I'll admit I
picked ecosystems without knowing the ratio of robust to fragile.)
> When one module fails (even a small one) typically the whole thing
> continues on as though nothing happened (aka log the error).
If the system can literally continue "as though nothing happened",
wouldn't it be a better design if we got rid of the module altogether ?
If your answer involves "the failure introduced an error that someone
will correct manually", then the system under consideration is not
limited to a collection of software modules (sphexish), it is software
plus people (flexible).
> This silly example isn't anything like the Arianne accident,
It's silly - in fact absurd, by design. Is it *entirely* unlike the
Ariane failure?
(We're venturing into Hofstadterian territory; it involves a crash and
misinterpreting something often used as diagnostic information. "Eating
a pumpkin", now there's something that isn't anything like the Ariane
crash; agree ?)
> make it slightly less silly - That's a bit like a car driver hearing a
> front passenger shriek "Oh no!", distractedly turning to look at the
> passenger, and crashing into the slowing traffic in front of them.
I'll call you on that. "Oh no" has different content - it signifies
urgent alarm. (At least the way I imagine your fictional passenger would
sound; maybe she's just saying "Oh no" in a disgusted tone, for all I
know. Your narrative would be a lot less credible.)
It's good design for a survival system to pay attention to signals for
*immediate* danger, temporarily ignoring ongoing peril. (People-in-cars
are less than perfect example of survival systems, for the reason that
they voluntarily choose to face ongoing peril...)
Laurent
Is there any chance of informed discussion when we are simply guessing
that the statements we make are correct?
My rather bad-tempered point was that if "you don't know the ratio of
robust to fragile" then why oh why are you claiming that "most
ecosystems can afford to lose one species".
> > When one module fails (even a small one) typically the whole thing
> > continues on as though nothing happened (aka log the error).
>
> If the system can literally continue "as though nothing happened",
> wouldn't it be a better design if we got rid of the module altogether ?
I just turned your statement around - I presume you have no better
basis for saying "when one module fails (even a small one) typically
the whole thing fails" than you did for commenting on ecosystems.
Is there some specific situation where you can say that you know "when
one module fails (even a small one) typically the whole thing fails"?
> Is there any chance of informed discussion when we are simply guessing
> that the statements we make are correct?
Sure, as long as they are informed guesses.
> My rather bad-tempered point was that if "you don't know the ratio of
> robust to fragile" then why oh why are you claiming that "most
> ecosystems can afford to lose one species".
There's a background rate of species extinction which I've seen quoted
at one to ten per year in prehistoric times, and I've not come across a
study of an extinction event or ecosystem collapse which attributes the
cause to loss of a single species.
Logically, it's also part of what we mean by "ecosystem"; an ecosystem
is a stable community of species, plus environment. Typically, the
identity and continued existence of a community does not crucially
depend on any single member of that community.
> Is there some specific situation where you can say that you know "when
> one module fails (even a small one) typically the whole thing fails"?
I'll retract "typically". Read "too bloody often" in its place.
You said: "When one module fails (even a small one) typically the whole
thing continues on as though nothing happened (aka log the error)."
If that was true, Ariane would certainly be *atypical*, in that the
computation which caused the exception was no longer relevant when the
program blew up. If it had logged the error and continued, rather than
shut the whole subsystem down, nothing would have happened.
Laurent
I don't really understand what that means:
- does "background rate" suggest that there's some other "species
extinction" going on which isn't included in the "background rate"?
- "one to ten per year" looks like a ball park estimate, which (if the
number of species has increased over time) seems to imply that
extinctions have decreased over time?
I can see that someone might have come up with something like that by
making both a low estimate and high estimate of extinctions over the
fossil record, and then working out an average per year.
If "one to ten per year in prehistoric times" is that kind of ballpark
average, then it doesn't seem to tell us anything about how those
extinctions were distributed in space or time. It doesn't seem to tell
us anything about how one species extinction may or may not have been
related to another.
> and I've not come across a study of an extinction event or ecosystem collapse
> which attributes the cause to loss of a single species.
Neither have I, but in my case it's simply because I haven't looked.
It would be more interesting to hear that you've looked at X out of N
studies of documented species extinctions and none of them
attributes...
> Logically, it's also part of what we mean by "ecosystem"; an ecosystem
> is a stable community of species, plus environment. Typically, the
> identity and continued existence of a community does not crucially
> depend on any single member of that community.
That just seems to be a different phrasing of the initial statement,
not an argument - it's begging the question.
> > Is there some specific situation where you can say that you know "when
> > one module fails (even a small one) typically the whole thing fails"?
>
> I'll retract "typically". Read "too bloody often" in its place.
So, somewhere between 1 and N where N>1
(And the specific situation would be - Ariane?)
> You said: "When one module fails (even a small one) typically the whole
> thing continues on as though nothing happened (aka log the error)."
>
> If that was true, Ariane would certainly be *atypical*, in that the
> computation which caused the exception was no longer relevant when the
> program blew up. If it had logged the error and continued, rather than
> shut the whole subsystem down, nothing would have happened.
I said "I just turned your statement around".
What is a "typical" software system?
> I don't really understand what that means:
> - does "background rate" suggest that there's some other "species
> extinction" going on which isn't included in the "background rate"?
The rate is much higher during extinction events, and during historical
times (apparently due to human intervention).
> > Logically, it's also part of what we mean by "ecosystem"; an ecosystem
> > is a stable community of species, plus environment. Typically, the
> > identity and continued existence of a community does not crucially
> > depend on any single member of that community.
>
> That just seems to be a different phrasing of the initial statement,
> not an argument - it's begging the question.
I'm not sure. Boiled down to essentials, what I'm adding to Thomas'
suggestion (software design should in the future be informed by how
humans deal with error and exception) is a suggestion that we compare
how "natural" and "designed" systems deal with error and exception. As
an example of "natural" systems, I point to ecosystems. As an example of
"designed" systems, I point to software. There are marked differences,
or so it seems to me, going on the information I have, in the way the
two classes of systems respond to the failure of one component.
Do you dispute that these differences exist ?
Laurent
"The rate is much higher during extinction events" by definition!
Again, this information is simply irrelevant to the question of whether
"most ecosystems can afford to lose one species".
It's a red herring.
Again, could the reason that "I've not come across a study of an
extinction event or ecosystem collapse which attributes the cause to
loss of a single species" - simply be lack of research?
Or to put it in more general terms "an inability to disprove does not
prove".
I'm sorry, these are not "informed guesses".
> > > Logically, it's also part of what we mean by "ecosystem"; an ecosystem
> > > is a stable community of species, plus environment. Typically, the
> > > identity and continued existence of a community does not crucially
> > > depend on any single member of that community.
> >
> > That just seems to be a different phrasing of the initial statement,
> > not an argument - it's begging the question.
>
> I'm not sure. Boiled down to essentials, what I'm adding to Thomas'
> suggestion (software design should in the future be informed by how
> humans deal with error and exception) is a suggestion that we compare
> how "natural" and "designed" systems deal with error and exception. As
> an example of "natural" systems, I point to ecosystems. As an example of
> "designed" systems, I point to software. There are marked differences,
> or so it seems to me, going on the information I have, in the way the
> two classes of systems respond to the failure of one component.
>
> Do you dispute that these differences exist ?
What do you imagine that "error" and "exception" and "failure" mean in
the context of an ecosystem?
What "purposes" do you imagine "ecosystems" pursue that could result in
"error" or "exception" or "failure"?
Hello Mr. Martin,
I recently come to know that you are the same person ,author of "C++
Gems" book
Can you plz give me more reference of LOP Online,Papers etc.
Regards
Raxit sheth
raxitsheth at yahoo.co.in
India