APL Program Code Metrics
Ibeam2000
of the new features is something called Code Metrics. In short, this
feature generates a statistic, a "maintainability index" (MI) for
every program.
The formula is something like
MI := 171
- (5.2 * ln(halstead volume))
- (0.23 * cyclical complexity)
- (16.2 * ln(lines of code))
This looks as if it would work for APL code with control structures.
(Cyclical complexity is basically a depth count of nested loops, if
statements, case statements, and so on) (How would one do this
meaningfully for legacy branching?)
Has anyone ever done anything like this for APL functions?
micr...@microapl.demon.co.uk
> Recently, one of my clients started using Visual Studio 2008, and one
> of the new features is something called Code Metrics. In short, this
> feature generates a statistic, a "maintainability index" (MI) for
> every program.
>
> The formula is something like
>
> MI := 171
> - (5.2 * ln(halstead volume))
> - (0.23 * cyclical complexity)
> - (16.2 * ln(lines of code))
>
> Has anyone ever done anything like this for APL functions?
Hmm, I would be very sceptical of anything like this, especially if a
manager uses it as a measure of the quality of a programmer's work.
To take some obvious examples of why this is unlikely to be of any
use:
- Two lines of code can be clearer and more maintainable than one.
This is especially true in APL, but to a lesser extent is true in
scalar languages as well. It all depends.
- Depth of nested loops is usually determined by the complexity of the
problem, not of the solution. Sure, in some cases in might be more
readable to break it up so that inner loops are placed in separate
functions, but in some cases it might not. It all depends.
- These kinds of metric incentivise the programmer to write lots of
very small functions, rather than fewer larger ones. But that's not
always a good thing. I've quite often been frustrated in trying to
understand C or C++ code because it is broken up into pieces which are
too small - you have to act as a human C compiler to follow the logic
(You're trying to understand a loop. It calls a function, so you save
your 'understanding stack' and go and look at that function. It does
nothing significant and then calls another function, so you save your
'understanding stack' again, and so on. Finally you come to a tiny
piece of actual code which could have been placed in the original
loop, where it would have been obvious what it did. Meanwhile you've
forgotten the original context.) So, again, it all depends.
Basically, there is no substitute for intelligent human judgement in
this. Some people write very clear and maintainable code, and some
don't. The style can vary from one good programmer to another good
programmer. In general, those who think clearly write clear code.
But if you DO want a metric, in my experience there's only one worth
having. I propose, free of charge and free of copyright, Nabavi's
Code Quality Metric, which applies to any language. It is C divided
by L, where:
- C is the number of lines with comments, excluding any boiler-plate
headers etc which can be pasted in from a template or generated
automatically;
- L is the number of code lines.
Anything over 1 is good for most languages. You may want a higher
value for APL, where one line of code can do so much.
Even this metric should be used with care, however, as it is only
valid if the comments are well thought-out, and explain what the
programmer thinks the program is doing. This is because, if you're
trying to find a bug in someone else's code, the main question you are
asking is: Does the code actually do, under all conditions, what the
programmer thought it does?
Richard Nabavi
MicroAPL Ltd
BTW this is Microsoft, right? The same people who produce a grammar-
checker (in Word) which is complete garbage? Try it on well-known
pieces of extremely clear and well-written English, such as
Churchill's speeches, and you'll see what I mean...
Ibeam2000
tightly controlled situations. For me, I need a rough idea of which
functions might need some attention.
The context where I would find this kind of metric userful is in a
maintenance situation when one is just given the task to maintain a
system containing of lots (thousands) of functions. I don't have
enough time to go through every function in detail. The code quality
can range from good to dismal to indescribably ghastly. The one
useful statistic, where the errors occur, is naturally missing. After
eliminating dead code (often code volume drops 20% or more), I now
need to know which functions to pre-emptively rewrite. If a function
is both "ghastly" and "problematic", I rewrite it, or more likely,
clean it up. Or "ghastly" and "slow", etc.
It's like a three-legged stool. Maintainability is only one dimension
of the problem. More important are reliability and speed.
I also feel this paricular metric, as it might apply for C#, is rather
naive for APL. Spaghetti code detection isn't exactly cyclomatic
complexity. Creation and use of global variables is simply not
addressed, and bad style does not necessarily make the code more or
less maintainable.
crishog
I wrote something years ago which did attempt to generate these
metrics. The intention was to give guidelines to the developer on the
"opinion" of the process on his/her code. It wasn't really meant for a
manager who would use it out of content.
As Richard says, it's entirely dependent on how you write your
solution to the problem - is the purpose of a piece of code a "once
off" (although those tend to hang around much longer than you expect -
"The service life of a cobbled up fix is inversely proportional to the
time required to slap it together." [Nick Lappos]) or is it a critical
part of a system which runs 24 hours a day in multiple locations? The
"Quality Priorities" which change vastly with different circumstances.
Having said that it IS possible to gather these metrics (the first
version was in APL*Plus for DOS back in the late 80s or early 90s), in
that we certainly achieved consistent results which we (generally)
agreed with - but we never showed the results to anyone outside of the
team :-)
Chris Hogan
micr...@microapl.demon.co.uk
>
> Having said that it IS possible to gather these metrics (the first
> version was in APL*Plus for DOS back in the late 80s or early 90s), in
> that we certainly achieved consistent results which we (generally)
> agreed with - but we never showed the results to anyone outside of the
> team :-)
>
I think that if you are trying to encourage (or force!) a particular
style for a particular type of software within a smallish team, then
such a metric makes more sense. But that's not the same as claiming
that the metric is some sort of objective measure of code quality,
programmer skill, or code maintainability, over a wide range of
different situations.
Richard Nabavi
MicroAPL Ltd
crishog
Lord no! I never try to force APLers to do anything: as John Jacob
says it's "like trying to herd cats".
All we were doing was trying to foster a house style across a team of
about 8-10, to make it a bit easier to jump into someone else's shoes
in an emergency. I don't think my code had anything like the rigour,
or even the sample size, to be able to call it objective.
We found it useful, because it provided a mechanism for evaluation. We
all look at other people code (and I hope our own) & think "that looks
good/bad". The metrics functions gave an impartial view, one effect of
which was more often to say the result was good enough, stop polishing
the code because you feel it's not neat enough & get on with the next
job. The reverse of "it's not good enough do it again".
Chris
Stephen Taylor <editor@vector.org.uk>
This thread encourages me in my view that writing software is more
like writing than it is like engineering.
"Program Code Metrics" sounds a reasonable subject for engineers to
study. Contemplate an equivalent for, say, novelists, and the eyes
pop.
This is not to argue that how one writes – that is, one's style of
writing – is any less important in writing software than in writing
novels. But it might discourage us from thinking it any less deep or
subtle a subject either.
Metaphors matter. This is another example of when it's more helpful to
think of ourselves as writers (describing imaginary machines) than
engineers.
Simple rules have their value, as they do in teaching English
composition to young children. But they will only carry one so far.
This is not to argue there is nothing to be said about style, only
that it is unlikely to be simple or categorical, any more than it is
for EngLit students.
I don't know anyone in our field who thinks writing style matters as
much to the lifetime cost of software as I think it does. (And no –
wry grin – I _don't_ have metrics for that.) But, just as in writing
English, I have never seen a rule it hasn't been profitable and
appropriate to break on occasion.
Persig, I think, got it about right in "Zen and the Art of Motorcycle
Maintenance". He remembers teaching English when younger. Plagued by
student demands for clear rules, he works his way to the conclusion
that Quality is irreducible. It can be studied, but not reduced to
anything else in the sense, for example, that chemistry reduces to
physics.
So paradoxical conclusion: writing style matters very much to
professional writers concerned with the cost and serviceability of
software, and merits deep study. For most purposes (think Hemingway,
not Henry James) good style looks disarmingly simple. But there is
little simple to be said about it.
Stephen Taylor
edi...@vector.org.uk
http://www.vector.org.uk/
http://aplwiki.aplteam.com/
crishog
readability index for English text, though what it would make of the
difference between Hemingway and Dickens, let alone Shakespeare, I
honestly don't know.
I quote one comment from a site which advocates their usage: "A
readability index score is not an exact science. For example, it does
not consider disposition (paragraphs) or actual content (are the words
from a specific domain?)." - Which sounds like the reservations
expressed here.
Now don't get me wrong. I think "Software Engineering" is fairly much
an oxymoron. It implies some solidity and consistency to the process
of writing software: Like building a bridge - and we know what
happened to the millennium bridge in London. Many managers are failed
developers: they don't really understand the art of programming & they
feel the need to have some sort of handle or measurement of what these
magicians are up to.
My brother is an engineer - a mechanical engineer that is. The
impression I get of how he works though is more of an artist hand-
crafting something rather than it being a coherent science. He says
that often, even when following all the rules precisely, they don't
know if a machine will work until it's been built. Hence the
recognized need for prototyping, at least in mechanical engineering:
and never get him started on the problems of taking a working device
measured in American units & trying to built the same thing in
Imperial or Metric measurements.
However, as Lord Kelvin said “If you cannot measure it, you cannot
improve it.” and undoubtedly the Flesch-Kincaid index does measure
some aspects of written English and Halstead et. al. do measure
something about code. Readability does aid maintainability, but so do
internal comments and, best of all, well written code. None of the
program metrics make one a better programmer, nor are they perfect
tools, but (Kelvin again) "The more you understand what is wrong with
a figure, the more valuable that figure becomes."
But I'd not thought before of extending the literary analogy: some
authors are, or were, masters of short stories (Poe for example),
others playwrights, or novelists. Perhaps the different styles of
developer should be recognized in this way. We all know someone who is
a better system programmer, or a support person or an applications
programmer, and this must of necessity be reflected in the coding
styles they use.
I'm beginning to ramble.
Chris
micr...@microapl.demon.co.uk
>
> But I'd not thought before of extending the literary analogy: some
> authors are, or were, masters of short stories (Poe for example),
> others playwrights, or novelists. Perhaps the different styles of
> developer should be recognized in this way. We all know someone who is
> a better system programmer, or a support person or an applications
> programmer, and this must of necessity be reflected in the coding
> styles they use.
>
I think that Stephen's point about programming being more like writing
than engineering is astute. Both Jane Austen and Hemingway wrote
excellent English, but I don't think any computer-calculated metric
would give both of them high marks.
The analogy goes further. To write clearly, you need to think
clearly, and understand your subject. I believe the same is true of
writing software. The actual style may vary, but clarity is all.
Programming 'bugs' are very often like poorly-written English
sentences; they arise because the author has not considered that the
text might be misconstrued.
What worries me is that trying to measure software quality by using
metrics is as absurd as saying that this:
"Five score years ago, a great American, in whose symbolic shadow we
stand today, signed the Emancipation Proclamation. This momentous
decree came as a great beacon light of hope to millions of Negro
slaves who had been seared in the flames of withering injustice. It
came as a joyous daybreak to end the long night of their captivity.
But one hundred years later, the Negro still is not free. One hundred
years later, the life of the Negro is still sadly crippled by the
manacles of segregation and the chains of discrimination."
is inferior to this:
"Five score years ago, a great American, in whose symbolic shadow we
stand today, signed the Emancipation Proclamation. This momentous
decree came as a great beacon light of hope to millions of Negro
slaves who the flames of withering injustice had seared. It came as a
joyous daybreak to end the long night of their captivity.
However, one hundred years later, the Negro still is not free. One
hundred years later, the manacles of segregation and the chains of
discrimination still sadly cripple the life of the Negro."
(The first is Martin Luther King's original text from the 'I have a
dream' speech. The second is what he would have said if he'd had to
put it through Microsoft's grammer checker.)
Richard Nabavi
MicroAPL Ltd
Ibeam2000
the opposite of Quality?
It is precisely this negative quality I would like to estimate and
track and measure about code. It's not about programmer quality, nor
programmer productivity, nor exploring why the code got to this
state. If anything, it's about labour relations.
Suppose I have a group, three or more, humans who are working on some
code base. Two people working together perhaps know each other and
their respective coding habits rather well. But with three or more,
invariably the individuals become more disconnected. All too often,
one hears the phrase "This code is $#@% !"
But, as a sort of social contract, if I can suggest to try keep the
FOO index up above (some arbitrary number), and, in our copious free
time, and armed with other metrics, such as frequency of use, examine
and clean up functions which score poorly, I would expect the
following to happen:
Expectations
[1] Better "teamwork", at least some reduced tension and stress
amongst the group members
[2] Irrespective of what the FOO index indicates, an increase in code
quality, which leads to;
[3] An increase in code maintainability, which leads to;
[4] An increase in code reliability, and finally;
[5] goto 1
> “If you cannot measure it, you cannot improve it.”
> "The more you understand what is wrong with a figure, the more valuable that figure becomes."
These are excellent points. The metric can and should be developed by
the group, and it should be constantly refined (tinkered with). But
most importantly, the metric should be a catalyst for change, and not
an absolute measure of anything.
Doug White
In article <12a7b6f6-4342-4159...@l42g2000hsc.googlegroups.com>, crishog <goo...@4xtra.com> wrote:
<snip>
>My brother is an engineer - a mechanical engineer that is. The
>impression I get of how he works though is more of an artist hand-
>crafting something rather than it being a coherent science. He says
>that often, even when following all the rules precisely, they don't
>know if a machine will work until it's been built. Hence the
>recognized need for prototyping, at least in mechanical engineering:
>and never get him started on the problems of taking a working device
>measured in American units & trying to built the same thing in
>Imperial or Metric measurements.
My favorite story about CAD for mechanical engineering was from a Viet
Nam war era article about the development of an automatic (machine gun)
grenade launcher for the US Navy. They were all very excited about their
new finite element analysis mechanical CAD tool. They carefully designed
& built the launcher with plenty of safety margin according to the stress
analysis computations. They fired one shell & it worked perfectly, as
did a second and a third individual shot. They then loaded the magazine
with three rounds, set it to automatic, and it tore itself to pieces.
The chief engineer examined the wreckage, beefed up everything that broke
by an amount his experience said should be adequate, and that was that.
Although it's perhaps a bit of a stretch to apply this as a metaphor for
some programming endeavours, it's not too far off the mark.
Doug White
Stephen Taylor <editor@vector.org.uk>
> Suppose I have a group, three or more, humans who are working on some
> code base. Two people working together perhaps know each other and
> their respective coding habits rather well. But with three or more,
> invariably the individuals become more disconnected. All too often,
> one hears the phrase "This code is $#@% !"
I'm inclined to agree with you and the Extreme Programmers about the
importance of this issue. A coding team is writing for each other's
eyes. Famously, the XP people think code quality so important that
no one may check in production code unless it's been written with at
least one other team member.
I'd take their view to be that the issue is too important to leave
to metrics.
Stephen Taylor
edi...@vector.org.uk
http://www.vector.org.uk
http://aplwiki.aplteam.com
Gosi
They each deliver 1 function every day.
Programmer A is working on project B.
After 50 days programmer A is also fixing 10 problems in project B
every day.
Programmer C is working on project D.
After 50 days programmer C is still only delivering 1 function per day
because there are no bugs in project D.
It is possible to claim that programmer A is more productive because
he is delivering 10 fixes and 1 new function every day.
I have seen similar cases over and over.
Programmer A is a hero because he is so good at fixing problems.
crishog
<StephenTaylorF...@googlemail.com> wrote:
> I'm inclined to agree with you and the Extreme Programmers about the
> importance of this issue. A coding team is writing for each other's
> eyes. Famously, the XP people think code quality so important that
> no one may check in production code unless it's been written with at
> least one other team member.
>
> I'd take their view to be that the issue is too important to leave
> to metrics.
>
But XP does try to measure such things as "project velocity" - and I
would have thought you could only do that by knowing something about
what the team has produced, other than the weight of the printout.
But:
On Jun 23, 10:03 am, Gosi <gos...@gmail.com> wrote:
> I have seen similar cases over and over.
> Programmer A is a hero because he is so good at fixing problems.
As I said in my first posting: "The intention was to give guidelines
to the developer on the "opinion" of the process on his/her code. It
wasn't really meant for a manager who would use it out of content."
These tools are best used within the team itself, to assist with
things like project velocity & the "social change" which seems to
evolve from an iterative quality improvement scheme. Though I seem to
be espousing them, I don't believe the metrics have universal
objective meaning.
Chris
phil chastney
IEEE Software had a few articles some years ago on function point
analysis (there may have been others, but I'm no longer a member)
one protagonist conceded that the analysis needed to be tuned not only
for the language in use (and APL was included in the survey), but also
for the analyst
so much for comparability!
I would have thought similar comments apply here - you may get
comparable results for scalar-processing imperative languages, but for
APL, SQL, ML, Prolog, Scheme, assembler...? I doubt it very much
as others have already pointed out, any results would surely be
comparable only within a very local radius
it's interesting that the formula doesn't appear to consider
object-orientation as being a significant factor
/phil
jk
"phil chastney" <phil.ha...@amadeus.munged.eclipse.co.uk> wrote in
message news:nwU7k.79009$tn2....@fe07.news.easynews.com...
> Ibeam2000 wrote:
>> Recently, one of my clients started using Visual Studio 2008, and one
>> of the new features is something called Code Metrics. In short, this
>> feature generates a statistic, a "maintainability index" (MI) for
[...]
>
> IEEE Software had a few articles some years ago on function point
> analysis (there may have been others, but I'm no longer a member)
>
> one protagonist conceded that the analysis needed to be tuned not only
> for the language in use (and APL was included in the survey), but also
[...]>
> /phil
oh, gosh, the fpa, the nightmare for an APL-er ...
I remember this from the early to half the 90ties; the programmers in my
office where idolated of that - (it was new! and they thought they had a
yardstick - for what??)
Even before they knew what fpa was it was done ... in APL. Really.
Ted Edwards
> This thread encourages me in my view that writing software is more
> like writing than it is like engineering.
I would say like a cross between writing and mathematics.
> "Program Code Metrics" sounds a reasonable subject for engineers to
> study. Contemplate an equivalent for, say, novelists, and the eyes
> pop.
Personally, I think it a good subject for a team of psychologists
provided there's enough of them that they never reach any conclusion.
Later in this thread someone brought up XP. I have yet to encounter a
piece of M$ software that I would consider reliable.
APL2 for OS/2 is probably the most reliable piece of software I have
encountered and I'd bet no program code metrics were used by the
development team.
Ted
mps...@windstream.net
At one point in time, I made the windows/OS2 choice. Chose windows,
because of
the groups I was teaching. Still have the system but it needs a bit
of fan replacement
cleaning, ... It's been rock solid. Not fancy; slow but I'm used to
running it and going
to bed. APL2 is what I think-in. I've never realized that I need
anything better.
Not saying OS2 wasn't better than win3.1; just sayin APL2 was the
key. Metrics, to me, is
an exercise for software pro's. Wish em well, but I know what
works.
For customers, I wrote code with comment of what the code was doing,
with ref's to the
data structure and the algorithms used. With the view that they (the
customer) could redo
the program in pascal or ... if they had need later. It worked.
Marv in Lex, KY
Rav
I worked at a place (an APL shop) where Programmer A, who didn't believe
in testing his stuff and therefore worked somewhat "faster," ended up
having to work a weekend making fixes to his code after an important
site hit the bugs. Programmer B, who also worked on the same project
but did test his stuff, didn't end up with any bugs to fix. Programmer
A ended up being recognized for going above and beyond the call of duty
because he sacrificed his weekend "for the good of the project." Even
got a monetary award, I believe. No recognition for Programmer B.
** Posted from http://www.teranews.com **
Gosi
The best programmers are intelligent and lazy.
They tend to make tools to assist in everything.
If you are a fast typist and always remember everything you end up
forgetting to put in important lines in the system.
Also the fast programmer is always fixing the system on the fly while
running the program and does not bother to fix small problems.
If you always start your program from the beginning and after fixing
something start again until it runs without problems you may be
considered lazy for not being a fast fixer.
The intelligent part can be good too but is far less important.
Intelligent and fast can be a deadly situation especially if you are
an APL programmer.
If you are only lazy and not very intelligent you are probably not an
APL programmer.
jk
"Gosi" <gos...@gmail.com> wrote in message
news:e384756e-f9e3-4b2c...@e53g2000hsa.googlegroups.com...[...]
> The best programmers are intelligent and lazy.
[...]
> If you are only lazy and not very intelligent you are probably not an
> APL programmer.
One important property for a good & productive programmer is to be a
robber, a smart robber.
Just like an intelligent robber he should ideally be crafty. Fastest
programming is done by trick & cheat.
crishog
> I worked at a place (an APL shop) where Programmer A, who didn't believe
> in testing his stuff and therefore worked somewhat "faster," ended up
> having to work a weekend making fixes to his code after an important
> site hit the bugs. Programmer B, who also worked on the same project
> but did test his stuff, didn't end up with any bugs to fix. Programmer
> A ended up being recognized for going above and beyond the call of duty
> because he sacrificed his weekend "for the good of the project." Even
> got a monetary award, I believe. No recognition for Programmer B.
Didn't whoever controlled the overtime budget think to ask why "A" was
working weekends? Did that just think hie has no social life? Some
metric might have helped to uncover the truth - such as "bugs per
week" in the developer's code - not all metrics are automatic, of
course, that would have to be compiled by someone other than the
developer - in whose interest it is to record "0".
I think think there is a place for metrics, but too rigid an approach
is like trying to compare someone filling in a tax return, all those
tick boxes & daft numbers, with someone trying to write a poem. You
can't judge by bulk or fixity of layout - the poem's probably a lot
shorter & the evidence of its production is largely invisible, but I
know which I would prefer to read.
Chris
jk
"jk" <aq...@planet.nl (remove the q's)> wrote in message
news:4861e921$0$6012$ba62...@text.nova.planet.nl...
In our pensionfund we once had a COBOL programmer who was very fast.
Once when inspecting the source code we came across potatoes, leek,
carots ATL. Apparantly "copied" from the software of a grocery wholesale
dealer.
The second fastest way is trial and error, that is when using an
interpreted language like APL.
Next comes dismissing half of the staff of programmers. In that way you
force the remaining ones to work as twice as fast.