One specific: he said the ALU should cook up a certification exam. He
understood that certification might be a joke in our domain, but that
nevertheless it was the kind of thing that would make Lisp look more
respectable to MegaCorp types.

He even seemed to think a simple timed on-line test would do the trick,
though how you stop cheating I do not know.

This was a new one on me, and was only one bloke's input. I scoff at
certification for programmers, but if it would be easy to implement and
if folks in tall buildings care (two big ifs) why not?

--

  How about just asking people to give a paper at a Lisp conference and
  making sure that the quality standards are sufficiently high?

  http://www.cs.ucsb.edu/~almeroth/conf/stats/

--
Erik Naggum, Oslo, Norway

I am not saying it is not a good metric, I think it is.  But that it is too
subject to outside influences for this purpose.

  I must admit to some ulterior motives.  First, if this was the requirement,
  I would get off the hook because I have given a paper at a Lisp conference.
  But (more) importantly, it would make a lot of people submit papers to Lisp
  conferences and thus would make the conferences more interesting and more
  frequent.  Fundamentally, I do not see the point with certification if it
  is a "selfish" measure, i.e., one where the community benefit of having one
  more certified programmer is negative, which it would be if the purpose was
  to make it easier for managers to replace one Common Lisp programmer with
  another or have more people compete for the same jobs.  If managers want
  that, they can have it, but giving it to them should benefit the community
  more than the particular manager.  Otherwise, managers get a strangle-hold
  on the market and will work hard to /lower/ the certification requirements
  so that they have more people to choose from and can lower their costs.
  The entrance fee to the Certified Common Lisp Programmer market should be
  that you have done something that clearly benefits the existing certified
  programmers, not just something that benefits the /future/ candidates.  The
  same rationale underlies the requirements to grant professional degrees.

--
Erik Naggum, Oslo, Norway

Is there a metacertification exam template? Do folks write code during
the exam (as in more than just a line or two)? It would be great fun
here on cll to brainstorm an exam if we knew what it should look like.

btw, the gent from the big company (i think it was a phone company) said
the certification thing itself might draw folks towards CL. A new exam
announced on SlashDot and to the IT press in general in and of itself
would make clear that the news of CL's death (in Wired?) was greatly
exagerated.

--

What would you think of a tiered approach, apprentice -> journeyman ->
master setup.  Where the journeyman and master grades would have the
requirement to help current advanced members.  I believe I am talking
about a guild or guildish thing.    

Also there are certian certs out there that having makes you more money,
CCIE comes to mind.  It is a very hard test, here is how it works:
1: take a written test
2: iff you pass then you go to cisco and:
        a: day 1 build a network
        b: if you built it right then day 2 fix it after they break it.
It is designed to fail people.

I guess the trick is not to let the managers have much, if any, say in
the matter.

Screw MegaCorp, I say, let them choke on their own vomit.

--

And as Erik pointed out the idea of a Java type cert is a generaly bad
idea.

As it stands, CL gets laughed out of the running in IT, because the
immune system identifies us as "not-ready-for-IT". Maybe if we adorn
ouselves with a few ITish proteins like certification the immune system
will think we are friend, not foe.

And yes, it was just one opinion from one IT exec who had come to the
Lisp conference ostensibly because one of his people was arguing for
Lisp on the next project. Come to think of it, if we won't listen to
him, who should we listen to, to get into F500? It would be nice to get
more suits into this thread, or at least input from folks here who have
worked in tall buildings.

I have, and I can't say I heard anyone talking about certification.
otoh, i did know one IT recruiter who thought some guy was a genius
because he had passed a wadge of MS exams in short order (an Asian!).

--

Any company that has the expertise to determine their true needs and assess
programmer competence doesn't need these tests.  Use of these tests is
an admission of managerial incompetence.  Who wants to work for such a company?
I might do so only out of sheer desperation.

Is it possible that companies use these tests because they just happen to measure
exactly the skills that the company needs?  No.  I have taken many tests
in my life, and done quite well on most all of them, and I have never seen one that
measures much of anything other than the ability to do well on that particular test.
I know that this is a trite observance, but that doesn't make it any less true.

Programmer certification tests engender a situation akin to that created by "school
accountability" in the US.  In an attempt to lift everyone up to a minimum standard of
competence, we tie funding to test results, and end up churning out droves of students that
have been taught nothing but how to pass the tests.  It sucks for everyone in the end.

Don't cave in to immorality out of expediency.

  The problem with all certifications is that the skills needed to get the
  certification are different from the skills needed in the job requiring
  the certification.  The question we should ask "does this correlate with
  what we actually need?" and allow ourselves to be surprised by the many
  unexpected things that do correlate.  The human brain is deficient in its
  lack of capacity to see how many small things work together.  We are very
  good at singling out things that are important and focus on that one
  thing, but lousy at keeping track of masses of smaller interests that
  work together to make change.  Usenet is an interesting experiment in
  this regard.  Some people are so unable to process more than one quality
  at a time that they have to /invent/ aspects in order to retro-support
  their favorite quality.  This is not just the massive stupidity and lack
  of intelligence it looks like, it is how people are naturally wired to
  deal with the world if they do not consciously override it by thinking.

  This is somewhat like voting for people to lead a country.  The United
  States is /really/ paying the price for its plurality system during this
  presidential period.  For some reason, how many people would like
  something the most is regarded as a reason to choose it.  I favor a
  system where the number of people who like something the /least/ is
  subtracted from the number of people who like it the most, or generally,
  a system where each candidate is given positive and negative scores in
  some small range (where the sum of the absolute value of all scores is
  constant or has a fixed upper limit) and those you feel nothing about
  gets zero or no vote at all.  The scores are simply summed and whoever
  gets the highest total score wins.  The purpose of the negative votes is
  to ensure that someone who may well be favored by the largest minority
  but is loathed by a larger group, perhaps even a majority, not get into a
  position where the majority would feel they were not heard and which
  would destabilize the entire system.  This would ensure that a candidate
  would want to get backers on issues, not just fans of their person (or to
  avenge their father), and would have to calculate the risk of offending
  some groups, not just run over them.

  Back to certification, the number of points at which you would have to
  score well to be a good candidate for a job is attempted destilled into a
  certification, which at best may be assumed to mean "above the baseline",
  but the result may well be as undesirable as making George W. Bush the
  Republican presidential candidate.  In general, I want examinations and
  tests to score negative for a wrong answer and zero for no answer, and if
  it were up to me, "I don't know" would be far more socially acceptable
  than "I guess".  But no such luck.  Even the business community favors
  people who make wrong decisions over those who try to figure out what the
  best thing is and effectively make a decision not to act.  But you get
  what you deserve when you operate that way.  Unfortunately, people who
  should not get a job in programming get one because of certification,
  people who should be kept as far away from money as possible run both
  Enron and WorldCom and Arthur Andersen into the ground, and people who
  should be kept as far as away from Washington D.C. as possible get into
  the White House instead of staying in Texas and lots and lots of people
  suffer worldwide.  Incompetence should be a criminal offence.  The core
  problem is that certification does not solve any big problems, only small
  ones, just as book-keeping and auditing does not keep people from being
  criminals.

--
Erik Naggum, Oslo, Norway

The latest example was actually a couple of weeks ago, relating to
starting a contract.  I "punted" on a couple questions, hacking a
timestamp into UTC because I never can remember without consulting
manuals how to do date computations in SQL.  What was particularly
interesting was that the fellow reviewing the results commented that
some of the answers were unconventional, but that people usually got
the questions downright /wrong/.

And there probably lies the other piece: A /good/ test is one where
you'll get some answers wrong, because it has some questions tough
enough to challenge everyone.  Which may be nicely contrasted with the
"certification" thing where 'it isn't a good certification unless
typical Asian "memorize 'til you drop" techniques can be used.'

  I think that should be a personal decision.  It would be better for all
  of us if we did not have to check with what "we" want before each of us
  can make a personal decision, and consequently it would be nice if those
  who do not want something at least do not block the way for those who do.

| I have taken many tests in my life, and done quite well on most all of
| them, and I have never seen one that measures much of anything other than
| the ability to do well on that particular test.  I know that this is a
| trite observance, but that doesn't make it any less true.

  Tests are not useful for what they measure, but for how what they measure
  correlates with other things.  If some foot size happened to correlate
  well with programmer proficiency, one could measure foot size and get
  high programmer proficiency for no better reason than that other people
  with the same foot size had high proficiency as programmers; one would
  not measure proficiency as such.  Even if foot size correlated weakly
  with programmer proficiency, like 75% chance of getting a good programmer
  with some foot size, it could still be more valuable than anything else
  that had a lower correlation coefficient.

--
Erik Naggum, Oslo, Norway

Others have pointed out that Lisp projects tend toward small numbers of
programmers as opposed to cadres of mercenaries that Java encourages.

I would argue that no arbitrary third-party certification test is going to give you
as high a correlation to the skills that you are looking for as a small panel of
competent people asking questions, face-to-face, that they think are relevant to
their needs.
Reliance on such a test simply shows that there is no one of suitable competence
in an organization to either judge the worth (or lack thereof) of such a test or
devise their own test that correlates better to their own needs.
It seems to me that reliance on mass statistical analysis in this case is losing
vis-a-vis common-sense interpersonal judgement and intuition.
But that's just my intuition.

|   This is somewhat like voting for people to lead a country.  The
|   United States is /really/ paying the price for its plurality
|   system during this presidential period.  For some reason, how many
|   people would like something the most is regarded as a reason to
|   choose it.  I favor a system where the number of people who like
|   something the /least/ is subtracted from the number of people who
|   like it the most, or generally, a system where each candidate is
|   given positive and negative scores in some small range (where the
|   sum of the absolute value of all scores is constant or has a fixed
|   upper limit) and those you feel nothing about gets zero or no vote
|   at all.  The scores are simply summed and whoever gets the highest
|   total score wins.

This is probably equivalent to a system in which every voter has a
fixed number of (positive) points to distribute among candidates as he
wishes: Just add a constant to the points awarded by each voter in
your system to see this. Such systems are prone the same kind of
paradoxes that plague all the more conventional systems in
existence. These paradoxes are similar to, but not the same as,
Arrow's theorem, which is often interpreted as saying that perfect
democracy is impossible.

Anyway, I think a better way to achieve what you want (though not
paradox free - no fair voting system can be paradox free) is the
single transferrable vote. Each voter ranks all candidates. If one
candidates is ranked first on more than half of the ballots, he
wins. Otherwise, the candidate who is ranked first on the smallest
number of ballots is thrown out of the race, all the ballots are
adjusted accordingly, and the procedure starts from the top. (If N
candidates are to be elected, the requirement for being elected is to
have at least a proportion 1/(N+1) of the votes. Also, the ballots
that helped elect a winning candidate are put back in the pile of
ballots, albeit with a reduced weight. This case gets a bit more
complex.) If the latest US presidential election had been run this
way, one can imagine many voters ranking the candidates Nader, Gore,
Bush. Presumably there would be enough Nader+Gore votes to stop Bush
from winning outright; then since Nader would probably have the
smallest set of votes, he would be out of the race, most of his votes
would have gone for Gore instead, and Gore would win.

With an election system like this, it would be impossible for the
candidate of the largest minority to win, if he is uniformly loathed
by all the other minorities.

To get this if not back on topic, or at least a bit less off topic,
some of the problems underlying the difficulties of designing good
voting systems could equally well apply to the case of selecting one
of three job applicants A, B, and C, where after a careful evaluation
one finds that A is better than B, B better than C, and C better than
A - if what you're doing is ranking them according to at least three
different criteria, then just counting by how many criteria each
applicant is ranked higher than each of the others. Say, by their
intelligence (to the extent that you can gauge it) you rank them
A-B-C; but by their knowledge you rank them B-C-A, while by their
reliability you rank them C-A-B. Now A beats B in two of the three
criteria, while B beats C and C beats A in the same way.

  This means that you need a lot more than your regular certification.
  Which is why I think giving a paper at a conference is a good start.

--
Erik Naggum, Oslo, Norway

  Not quite.  There are two differences.  The first is what the absence of
  a vote means.  In a system with negative scores, absence means zero.  In
  a system with a skewed scale, absence of votes is generally not tolerated
  and voters have to give scores to every candidate.  This is known to fail
  miserably because once you ask people to rate things below the "don't
  care" limit, their score values are completely random.  It is therefore
  important to let voters decide not to score a particular candidate, and
  that the system give such a zero value, which is the second difference.
  The Borda system, which gives more points to the most values and zero to
  the least valued specifically requires that each candidate gets scored.

| Such systems are prone the same kind of paradoxes that plague all the
| more conventional systems in existence.

  The no-vote-means-zero-score rule removes a significant number of
  problems, but not all.

| Anyway, I think a better way to achieve what you want (though not paradox
| free - no fair voting system can be paradox free) is the single
| transferrable vote.  Each voter ranks all candidates.

  This is a huge problem.  Voters must be allowed /not to care/ about the
  relative ranking of candidates on whom they have no opinion.  Forcing
  voters to care about these candidates is known to produce lots of noise.

  Is this article you have read?  http://www.sciencenews.org/20021102/bob8.asp

--
Erik Naggum, Oslo, Norway