Erlang jobs
Joe Armstrong
<< I'm aware that this is a horrible breach of netiqueete but ...
it's not that often that you get to see ads. for functional programmers.
It's also a counter argument (by existence) to the (crazy) idea
that "industry isn't interested in FP" >>
so ...
+--------------------+
| Bluetail is hiring |
+--------------------+
Don't waste your brain come and work us ....
The story so far ...
Bluetail does stuff in Erlang - we believe we can do stuff faster
in Erlang than any other way - we did some stuff and then ...
We got bought up by Alteon << who has been bought up by Nortel >>
This means that the stuff we do will be sold by *lots* of salespeople.
This means we get an opportunity to change the (programming) world ....
and so we want to employ more "development engineers" - << people
who can do stuff that we can sell :-) >> to join in the fun.
How we work
-----------
Get stuff done
Have some fun
Make some money
Make an impact
We:
1) Have idea
2) Try the idea out on some customers
3) Implement it (yourself or with help)
4) Deliver to trial customers
5) If the trial customers like it get zillions of salespeople to sell it
or back to 1)
Note that:
- the cycle time here is very short (think months - not years)
- You will be expected to implement and sell your own ideas, or
you may end up helping somebody else implement their ideas
- Non profitable project will be ruthlessly cut
Who we want
-----------
- Brilliant programmers (must know C, Erlang, ... Unix)
- Pragmatists (We *really try* to delivery most stuff in Erlang, but if this
is too slow, we'll happily drop Erlang and do it in C. We even write
Java if the customers want it :-)
- Evangelists who get stuff done
We believe in FP (and Erlang in particular).
We believe that we can do stuff faster and better in Erlang than
any other language.
- People with lots of "hands on computing experiences",
typical everyday jobs might be:
- rig Linux on some weird hardware
- write a new emacs mode
- read 25 RFCs and figure out what the "real problem" is
- talk to a customer
- give an Erlang course
- Fly to the states with 24 hours notice
- ...
Extract pluses
---------------
Non of these are essential!
- In depth knowledge of wireless stuff
(WAP, Imode, ...)
- In depth knowledge of distributed programming or programming
embedded highly available systems
- Knowledge of protocols (TCP/IP, IPv6, ICQ, Freenet ...)
- Speak Swedish
Benefits
--------
- Money (amazingly - we pay you (you mean you get paid to have fun?
- yes ! :-))
- You'll learn a lot (not theory but "learning by doing")
/Joe
--
Joe Armstrong,
Bluetail AB, tel: +46 8-545 550 00
S:t Eriksgatan 44, fax: +46 8-545 550 50
SE-112 32 Stockholm, Sweden info: www.bluetail.com
Jeffrey Straszheim
> Don't waste your brain come and work us ....
<snip, snip, snip>
That sounds like the greatest job ever :)
However, I think if my wife heard me utter, "There is this great job
in Sweden", she'd immediately strangle me.
Looking around, there is nothing in my area other than Java,
Microsoft-Visual-Whatever jobs, and stuff of that nature. Sometime it
is depressing. I consider myself lucky that I found a job where I get
to use Linux and Digital Unix all day, albeit writing e-commerce crap
in Java.
*sigh*
--
-- Jeffrey Straszheim | A sufficiently advanced
-- Systems Engineer, Programmer | regular expression is
-- http://www.shadow.net/~stimuli | indistinguishable from
-- stimuli AT shadow DOT net | magic
Bruce Hoult
<Ulf.Wiger.c...@ericsson.com> wrote:
> I think it's mostly the art of communication that lags behind.
> You simply cannot achieve the same kind of productivity in innovation
> when you're geographically separated with today's methods of
> project management and software production. The limitations on
> today's technology play a part.
Simply avoiding those all-afternoon meetings ups distributed
productivity a lot.
> The shorter your turnaround times, the more you will
> hurt from distributed development -- esp. if you distribute over
> multiple different timezones.
I'm not sure that's true, on several counts.
Given tools such as ICQ, AOL/Netscape Instant Messenger or a MOO, you
can communicate pretty effectively over any distance. We used a MOO to
coordinate our "Dylan Hackers" ICFP entry (along with cvs and
occasionally email, an ad-hoc web page, or a SMS message to an absent
team member's cellphone) and it worked fine. Even when I've been
working on a contract in cubical-land in the last several years it's
very very common to have quick question/answer sesions on Instant
Messenger with the guy three cubes over.
I'd also dispute the timezone thing. I think it's a *big* advantage to
have timezones that don't overlap too much. Once again, on my forays
into cubical-land, I've found that the best productivity has come when
I've been working closely on something with someone else and they've
been a "morning person" who likes to come into the office at 6 am and
leave at 2 pm, while I wander in at midday and leave at 8 pm. There is
still several hours to coordinate things, but you get a lot of
contiguous time without interruptions to implement some part of the
puzzle and make it available to the other guy for when he next arrives.
And, again, I think that worked very well on our recent ICFP entry and
you can't get much shorter turnaround than that. With people in NZ, DE
and east coast US we had time differences of 8 hours, 6 hours, and 10
hours and it worked great. I had time to completely rejig our
matrix/vector code while the guy in the US was asleep. The guy in DE
dropped the compiler to us at the end of his day, and by the time he was
awake again we'd integrated it with the lexer and the renderer.
> For example, doing development both in the U.S. and Sweden, you have
> to suffer the fact that you have only a few hours (if that) when
> normal working hours overlap. This means that all situations where
> direct communication is essential will suffer significant delays.
And there have been studies that have shown a big productivity increase
with teams at a single site when they have set aside several-hour "don't
bug me" periods during the day and restricted communication to defined
times.
-- Bruce
Joe Armstrong
> Jeffrey Straszheim wrote:
> > On 01 Sep 2000 14:19:32 +0200, Joe Armstrong .. wrote:
> >
> > > Don't waste your brain come and work us ....
> >
> > <snip, snip, snip>
> >
> > That sounds like the greatest job ever :)
It is *believe* me :-)
>
> Don't get too excited about it: most job
> offers sound that way. (Now I feel an urge
> to write something about appearance vs.
> reality, but this is comp.lang.functional
> not talk.philosophy.)
>
> > However, I think if my wife heard me utter,
> > "There is this great job in Sweden", she'd
> > immediately strangle me.
Shame. It's nice here! - think of all those poor Swedes who have to go
the states :-)
>
> Why is it that you should be required to move
> to Sweden just to develop software? Why is it
> that software development still hasn't managed
> to go virtual? - I am no Marxist, but, I suppose,
> Marx would have argued that this is because the
> 'relations of production' are sadly lagging behind
> the state of the technology.
>
Well this is because you *can't* develop SW in distributed groups and
use e-mail (or whatever) to glue the development together.
I *strongly* believe that the only way to do large SW projects (Large
equals > 1 programmer) is to have all the programmers working in the
same corridor - even then it's very difficult and there will be a lot
of misunderstandings.
The key problem in software development is that different members of
the group do not really understand each other. Getting them to conduct
all conversations through e-mail is a recipt for disaster.
--
Luke Gorrie
> > However, I think if my wife heard me utter,
> > "There is this great job in Sweden", she'd
> > immediately strangle me.
>
> Why is it that you should be required to move to Sweden just to
> develop software? Why is it that software development still hasn't
> managed to go virtual? - I am no Marxist, but, I suppose, Marx would
> have argued that this is because the 'relations of production' are
> sadly lagging behind the state of the technology.
My experience from telecommuting at Bluetail from Australia is that
it's not the right way to do things :-) We have a very effective
discussion-in-the-corridoor etc sort of communications system that you
really need to be in, otherwise you're just not in the loop. If you
need to discuss a design, you can just grab a couple of people in the
corridoor and sort it out right there, which is much nicer than trying
to formulate a clear and precise email (or UML diagram ;-) for someone
to read when they wake up in another time zone.
I've also telecommuted at a telecommuting-centric company before,
which had emphesis on email and ICQ-esque chat and that sort of thing
coordinated over a bunch of time zones. I think communicating in that
way over a long period of time was rather painful and cumbersome, even
while working on relatively isolated programs and with only a handfull
of people.
My advice: if you want to work at Bluetail, come to Sweden :-)
Cheers,
Luke
Ian Grant
Joe Armstrong <j...@bluetail.com> writes:
>Well this is because you *can't* develop SW in distributed groups and
>use e-mail (or whatever) to glue the development together.
I'm amazed to hear this. What about, say, the Linux kernel or any one of
dozens of other successful distributed development efforts? You must have
intended to qualify this claim and I'm genuinely intrigued to know how.
--
Ian
Michael Schuerig
> Michael Schuerig wrote:
> > Bruce Hoult <br...@hoult.org> wrote:
> >
> > > In article <1eges6k.m991tqdxdcu0N%schu...@acm.org>,
> schu...@acm.org
> > > (Michael Schuerig) wrote:
> > >
> > > > Obviously the development model you describe works for the
> > > > teams you've been on. Are you sure it can be adopted by every
> > > > team? I have this suspicion, that it depends very much on the
> > > > people involved: Highly skilled and highly dependable.
>
> But who says that skill level and dependability are
> *fixed* traits? - Keep them dependent, and they will
> become neither independent nor dependable. Keep them
> doing mindless busywork, and they won't improve their
> skill level.
Sounds good, but doesn't fit what I see in practice. If met only very
few software people who actually spend time furthering their knowledge
and skills beyond what is minimally required by the job at hand. Nothing
wrong with it, if that's what these people want and like. It's just not
what I want.
Michael
--
Michael Schuerig
mailto:schu...@acm.org
http://www.schuerig.de/michael/
Luc Taesch
> > >
> > > That sounds like the greatest job ever :)
>
> It is *believe* me :-)
> >
no offense intented, but i heard tax in danamrk is about 70%. what about
sweden ?
Christian Stapfer
> Christian Stapfer ... writes:
>
> > Jeffrey Straszheim wrote:
> > > On 01 Sep 2000 14:19:32 +0200, Joe Armstrong .. wrote:
> > >
> > > > Don't waste your brain come and work us ....
> > >
<snip/>
> >
> > Why is it that you should be required to move
> > to Sweden just to develop software? Why is it
> > that software development still hasn't managed
> > to go virtual? - I am no Marxist, but, I suppose,
> > Marx would have argued that this is because the
> > 'relations of production' are sadly lagging behind
> > the state of the technology.
>
> Well this is because you *can't* develop SW in distributed
> groups and use e-mail (or whatever) to glue the development
> together.
Such development has already been done (the X Window
system was an early example), thus it's no use arguing
that it can't be done in principle, as you do. The only
point that remains open for discussion is how *effective*
distributed development is likely to be as compared
to having everybody 'under the same roof'.
The 'space' separating the various minds cooperating
in the construction of a software system just might be
mostly of the mental, and not at all of the physical kind:
if it is, then no exertion of the *legs* is will bring
those minds nearer to one another.
> I *strongly* believe that the only way to do large SW
> projects (Large equals > 1 programmer) is to have all
> the programmers working in the same corridor - even then
> it's very difficult and there will be a lot of misunderstandings.
To call a SW project large just because more than
*one* programmer happens to work on it, as you do,
seems quite implausible to me. (Try posting your
definition of 'large project' to
news:comp.software-eng, they are likely to think
it splendid fun, a good joke...)
There is a marked difference between a project
done by, say, 2 people and 20, for example. Your
definition of 'large project' simply sweeps that
difference under the rug. It is exactly with medium
to large projects that communicating over the corridor
is not such a wonderful thing to do. - Small projects,
on the other hand, are not a problem anyway.
It is perhaps a mark of our time that communication
is unthinkingly taken to be a Good Thing. However,
you would not want to have Jack and Jill exchange
too many details about the current state of the
implementation of their components, since this
might cause them to break the interfaces of those
components, however unwittingly (for example,
knowing implementation details they may assume
certain performance characteristics that are
not guaranteed by the interface. E.g: "He told
me that he wants to use a hash table, hence
this operation will be O(1)".).
Similarly, if your project is going to be
sufficiently large, you must see to it that changes
are made in clear, discrete, carefully synchronized
increments: Frederick P. Brooks had to learn that
lesson back in the 1960s the hard way - are we now
forgetting it again?
If anything is a recipe for disaster it is
quickly and chaotically communicating change
requests over that wonderful corridor of yours...
Another advantage of distributed development is
that people will not just shout across the room
- or across the corridor - to advertise an idea
or a problem, but will first try to put it into
words rather more carefully.
Trying to write things down helps not only
communicate what the writer himself already knows
(and distribute it much more effectively than your
small-talk-distribution corridor ever could) -
it quite often helps him clarify his own thought
*before* he actually communicates it; thus reducing
the cognitive load of everyone else working on the
project considerably.
Further: written communication persists, it can
travel through time, can be read asynchronously,
can be copied and pasted, can be cross-linked,
can be edited, and can be re-read to refresh and
correct our all too fragile and distorting human
memory. As Larry Wall put it wittily:
"Laziness: The quality that makes you go to
great effort to reduce *overall* energy expenditure.
It makes you ... *document* [the program] you
wrote so you don't have to answer so many questions
about it." (my emphasis)
Having Larry distribute his knowledge of Perl,
say, over the corridor only, would not at all
be effective: it would limit the number of
receivers of that knowledge drastically and,
worse still, it would be too much a drag on
Larry's valuable time.
Being persistent, written communication has
the advantage of producing a log of the flow
of ideas, problems and changes as well. Such a
log is as useful for humans as it is for many
software systems. For example, if the meta-system
is to really *learn* from its mistakes, it would
be a good thing to do careful 'post mortem' analysis
of a failed project. Such post mortem analysis
*requires* a written trace. (There is nothing
wrong with analyzing a successful project with
equal care, of course, though we wouldn't call
it doing a post mortem... )
> The key problem in software development is that
> different members of the group do not really
> understand each other. Getting them to conduct
> all conversations through e-mail is a [recipe] for
> disaster.
Agreed, *if* they haven't managed to make it
beyond what cultural anthropologists would
call 'primary orality'. Literacy is, of course,
a requirement for distributed software development.
But isn't it a requirement anyway? - If they can't
put their thoughts into succinct prose, it seems
unlikely tom me that they can design a medium to
large system properly anyway.
You can't keep the whole corridor of people
as a maintenance system either, thus the
communication structure of the object-system
has to be *far* simpler than that of a hotchpotch
of cross-communicating components that simply
reflects the communication structure of an
equally chaotic meta-system (your corridor).
To summarize, I believe that your corridor-system
is likely to work best with a large number of small
projects, not at all with a small number of medium
to large projects, as you seem to suggest.
Christian Stapfer
-----------------
»I think by writing.«
- Paul R. Halmos: 'I Want to be a Mathematician'
»A thought written down (and not immediately thrown
into the wastebasket) is stubborn, doesn't change
its shape, can be compared with the other thoughts
that come after it.«
- Howard S. Becker: 'Writing for Social Scientists'
»Writing fosters abstractions that disengage knowledge
from the arena where human beings struggle with one
another. It separates the knower from the known. By
keeping knowledge embedded in the human lifeworld,
orality situates knowledge within a context of
struggle.« - Walter J. Ong: 'Orality and Literacy'
Joe Armstrong
I guess I shouldn't have said "can't" because you obviously can - but
in my experience it's exceedingly inefficient.
Luc Taesch
> ig...@cl.cam.ac.uk (Ian Grant) writes:
>
> > In article <lwem2zq...@pippi.bluetail.com>,
> > Joe Armstrong <j...@bluetail.com> writes:
> > >Well this is because you *can't* develop SW in distributed groups and
> > >use e-mail (or whatever) to glue the development together.
> >
> > I'm amazed to hear this. What about, say, the Linux kernel or any one of
> > dozens of other successful distributed development efforts? You must have
> > intended to qualify this claim and I'm genuinely intrigued to know how.
> >
>
> I guess I shouldn't have said "can't" because you obviously can - but
> in my experience it's exceedingly inefficient
mayber its razher a matter of time, rather than success.
it takes more time to communicate, and more effort, as written is slower than
speaking, but then gives u more time to think.
so , under prresure to deliver, its probabaly not the best way...
also, another factor is the interactivity. sometimes, i grow ideas just by
reacting on on other people ideas, say them , and other in turn react. in
fact, my best ideas, have been created under reaction to others conservative
viewpoints (which i counteract with a "progressist" viewpoint, on the spot)
this kind of sponteanous behavior is hader by email/icq of course.
> .
>
> /Joe
> --
> Joe Armstrong,
> Bluetail AB, tel: +46 8-545 550 00
> S:t Eriksgatan 44, fax: +46 8-545 550 50
> SE-112 32 Stockholm, Sweden info: www.bluetail.com
--
First, they ignore you.
Then, they laugh at you.
Then, they fight you.
Then, you win.
--- Gandhi.
Working code is what matter, not your market capitalization.
--Kurt granroth
Christian Stapfer
> Christian Stapfer .. wrote:
>
> > Michael Schuerig wrote:
> > >
> > > Bruce Hoult .. wrote:
> > >
> > > > In article <1eges6k.m991tqdxdcu0N%schu...@acm.org>,
> > schu...@acm.org
> > > > (Michael Schuerig) wrote:
> > > >
> > > > > Obviously the development model you describe works
> > > > > for the teams you've been on. Are you sure it can
> > > > > be adopted by every team? I have this suspicion,
> > > > > that it depends very much on the people involved:
> > > > > Highly skilled and highly dependable.
> >
> > But who says that skill level and dependability are
> > *fixed* traits? - Keep them dependent, and they will
> > become neither independent nor dependable. Keep them
> > doing mindless busywork, and they won't improve their
> > skill level.
>
> Sounds good, but doesn't fit what I see in practice. If [=I have?]
> met only very few software people who actually spend time
> furthering their knowledge and skills beyond what is minimally
> required by the job at hand. Nothing wrong with it, if that's
> what these people want and like. It's just not what I want.
>
> Michael
Well, yes, that's exactly what I meant to argue:
if the *job* does not require it, people tend not
to increase their skill level. Worse still, many
are kept doing mindless busywork that prevents
them from even exercising those skills that they
already have acquired. But if they can't exercise
a complex skill sufficiently they might lose it.
Sadly, keeping skill level low is sometimes in
the interest of management since it ensures that
workers remain easily exchangeable cogs in their
profit-making machine.
Of course, there just might exist exceptional
individuals who keep learning and improving their
skills even under *unfavourable* conditions. The
example of Einstein working as a patent examiner
comes to mind - but that was back in the 1910s,
long before the apotheosis of Shareholder and
Manager...
Christian Stapfer
-----------------
»The full expression of the worker's skill conflicts
directly with the needs of management.«
- Harley Shaiken: 'Craftsman into Baby Sitter'
Joe Armstrong
> >
> > Well this is because you *can't* develop SW in distributed
> > groups and use e-mail (or whatever) to glue the development
> > together.
>
> Such development has already been done (the X Window
> system was an early example), thus it's no use arguing
> that it can't be done in principle, as you do. The only
> point that remains open for discussion is how *effective*
> distributed development is likely to be as compared
> to having everybody 'under the same roof'.
Yup - it is entirely a question of effectivness -IHMO the
effectiness drops dramatically when you physically separate the
people doing the development.
> The 'space' separating the various minds cooperating
> in the construction of a software system just might be
> mostly of the mental, and not at all of the physical kind:
> if it is, then no exertion of the *legs* is will bring
> those minds nearer to one another.
>
> > I *strongly* believe that the only way to do large SW
> > projects (Large equals > 1 programmer) is to have all
> > the programmers working in the same corridor - even then
> > it's very difficult and there will be a lot of misunderstandings.
>
> To call a SW project large just because more than
> *one* programmer happens to work on it, as you do,
> seems quite implausible to me. (Try posting your
> definition of 'large project' to
> news:comp.software-eng, they are likely to think
> it splendid fun, a good joke...)
Large = "more than one headfull"
> There is a marked difference between a project
> done by, say, 2 people and 20, for example. Your
> definition of 'large project' simply sweeps that
> difference under the rug. It is exactly with medium
> to large projects that communicating over the corridor
> is not such a wonderful thing to do. - Small projects,
> on the other hand, are not a problem anyway.
>
I see big differences between
o one man project (easy to manage :-)
o 1-6 man projects (doable - but difficult)
o > 6 man projects (uuugh)
> It is perhaps a mark of our time that communication
> is unthinkingly taken to be a Good Thing. However,
> you would not want to have Jack and Jill exchange
> too many details about the current state of the
> implementation of their components, since this
> might cause them to break the interfaces of those
> components, however unwittingly (for example,
> knowing implementation details they may assume
> certain performance characteristics that are
> not guaranteed by the interface. E.g: "He told
> me that he wants to use a hash table, hence
> this operation will be O(1)".).
> Similarly, if your project is going to be
> sufficiently large, you must see to it that changes
> are made in clear, discrete, carefully synchronized
> increments: Frederick P. Brooks had to learn that
> lesson back in the 1960s the hard way - are we now
> forgetting it again?
> If anything is a recipe for disaster it is
> quickly and chaotically communicating change
> requests over that wonderful corridor of yours...
I have worked in a number of different environments and feel quite
at home in the chaotic environment. Strangely these chaotic environments
produced high quality software - on time, on budget and with very few
errors
I have also worked in environments with strict change control -
here the results were not so good.
I have a feeling that any simple problem can be made arbitrary
difficult by imposing a suitably heavy administrative process around
the development.
>
> Another advantage of distributed development is
> that people will not just shout across the room
> - or across the corridor - to advertise an idea
> or a problem, but will first try to put it into
> words rather more carefully.
> Trying to write things down helps not only
> communicate what the writer himself already knows
> (and distribute it much more effectively than your
> small-talk-distribution corridor ever could) -
> it quite often helps him clarify his own thought
> *before* he actually communicates it; thus reducing
> the cognitive load of everyone else working on the
> project considerably.
Writing is fine - but it is not good for explaining things to people.
When I explain things to people I watch their faces to see if they
understand or not. If they do not understand I change the explanation
so that they do understand.
That's why personal teaching is much better than reading things from
books - the book is stupid and does not realize that the reader
doesn't understand the content.
Also the *quality* of written matter in big projects often leaves much
to be desired. How in 10 GBytes of framemaker documents can I find
the relevant documentation. More often than not the written documents
make no sense.
> Further: written communication persists, it can
> travel through time, can be read asynchronously,
> can be copied and pasted, can be cross-linked,
> can be edited, and can be re-read to refresh and
> correct our all too fragile and distorting human
> memory. As Larry Wall put it wittily:
> "Laziness: The quality that makes you go to
> great effort to reduce *overall* energy expenditure.
> It makes you ... *document* [the program] you
> wrote so you don't have to answer so many questions
> about it." (my emphasis)
> Having Larry distribute his knowledge of Perl,
> say, over the corridor only, would not at all
> be effective: it would limit the number of
> receivers of that knowledge drastically and,
> worse still, it would be too much a drag on
> Larry's valuable time.
Yes - good books contribute a lot.
Unfortunatly big projects tend to "drown in paper"
> Being persistent, written communication has
> the advantage of producing a log of the flow
> of ideas, problems and changes as well. Such a
> log is as useful for humans as it is for many
> software systems. For example, if the meta-system
> is to really *learn* from its mistakes, it would
> be a good thing to do careful 'post mortem' analysis
> of a failed project.
In the face of failed projects there are strong political forces
that do not want a 'post mortem' analysis.
In the case of successful projects all the good stuff is wrapped
up in patents copyrights and legal stuff to make sure that people
*don't* learn how it was done!
> Such post mortem analysis
> *requires* a written trace. (There is nothing
> wrong with analyzing a successful project with
> equal care, of course, though we wouldn't call
> it doing a post mortem... )
>
> > The key problem in software development is that
> > different members of the group do not really
> > understand each other. Getting them to conduct
> > all conversations through e-mail is a [recipe] for
> > disaster.
>
> Agreed, *if* they haven't managed to make it
> beyond what cultural anthropologists would
> call 'primary orality'. Literacy is, of course,
> a requirement for distributed software development.
Yes - I agree - but being able to write well seems
to be a skill that many programmers have not mastered.
> But isn't it a requirement anyway? - If they can't
> put their thoughts into succinct prose, it seems
> unlikely tom me that they can design a medium to
> large system properly anyway.
And they don't. Very few people can put their thoughts into
succinct prose - and very few people can write good code.
Most programmers would be well advised to follow
Orwell's advice in his "Politics and the English Language"
but they don't (sigh :-()
> You can't keep the whole corridor of people
> as a maintenance system either, thus the
> communication structure of the object-system
> has to be *far* simpler than that of a hotchpotch
> of cross-communicating components that simply
> reflects the communication structure of an
> equally chaotic meta-system (your corridor).
> To summarize, I believe that your corridor-system
> is likely to work best with a large number of small
> projects, not at all with a small number of medium
> to large projects, as you seem to suggest.
It works very well for small projects.
As a computer scientist I hope that we can make powerful languages
so that complex products can be made by small "corridor" teams.
How to do "big projects" is still an open problem - I have seen
nothing that encourages me here.
> Christian Stapfer
> -----------------
> »I think by writing.«
> - Paul R. Halmos: 'I Want to be a Mathematician'
>
> »A thought written down (and not immediately thrown
> into the wastebasket) is stubborn, doesn't change
> its shape, can be compared with the other thoughts
> that come after it.«
> - Howard S. Becker: 'Writing for Social Scientists'
>
> »Writing fosters abstractions that disengage knowledge
> from the arena where human beings struggle with one
> another. It separates the knower from the known. By
> keeping knowledge embedded in the human lifeworld,
> orality situates knowledge within a context of
> struggle.« - Walter J. Ong: 'Orality and Literacy'
>
>
>
>
--
George Russell
> I'm amazed to hear this. What about, say, the Linux kernel or any one of
> dozens of other successful distributed development efforts? You must have
> intended to qualify this claim and I'm genuinely intrigued to know how.
(1) There wasn't any need for meetings or any form of co-ordination to specify
the general outline, because Linux follows Unix fairly closely, and doesn't
need any big new general design decisions.
(2) A large proportion of the Linux code could be developed by a single developer
who wanted to make an improvement which didn't impact so much on everyone else.
For example, by adding a device driver, or improving the algorithm for handling
page-faults.
I suspect the Linux model of development would be much harder if Linux did something
really radical. GNU Hurd seems to be more innovative.
Ketil Z Malde
> Yup - it is entirely a question of effectivness -IHMO the
> effectiness drops dramatically when you physically separate the
> people doing the development.
Effectiveness might drop, due to communication overhead. OTOH, as
somebody pointed out, lots of communication might lead to shortcuts,
interface breakage, poor documentation.
If the effort is distributed, there is more weight on clear interfaces
and good (and accurate) documentation, since development to a much
larger degree depends on it.
IMHO.
> Large = "more than one headfull"
There are many situations that come up. E.g. a new programmer
has to work on old code: having somebody in the corridor who's been
working on it for ages is a great benefit. Of course, the benefit
increases with the crappiness of the code - clean, well documented
code won't be as affected as undocumented spaghetti.
> I see big differences between
> o one man project (easy to manage :-)
> o 1-6 man projects (doable - but difficult)
> o > 6 man projects (uuugh)
My impression is that the Linux kernel is fairly well modularized into
1 or 1-6 man projects, with clearly defined interfaces. Which is
probably a necessity for a distributed effort to work.
Anyway, a six man project isn't that large IMHO, and I think corridors
scale less than well.
>> If anything is a recipe for disaster it is quickly and chaotically
>> communicating change requests over that wonderful corridor of
>> yours...
> I have worked in a number of different environments and feel quite
> at home in the chaotic environment. Strangely these chaotic environments
> produced high quality software - on time, on budget and with very few
> errors
> I have also worked in environments with strict change control -
> here the results were not so good.
So your corridors don't use change control, requirements,
specifications or other documentation procedures? IMLE, these are
really important for large projects.
> I have a feeling that any simple problem can be made arbitrary
> difficult by imposing a suitably heavy administrative process around
> the development.
The problem IMHO is that engineers don't understand the point of
development documentation, and thus ignore them. Or fill them with
meaningless words, which is even worse.
People need to understand that documentation has a purpose, and what
that purpose is, and whom the intended audience is, before they write it.
> Writing is fine - but it is not good for explaining things to people.
So how do you get somebody to explain a piece of code, if that person
quit the company four years ago? Or gets run over by a bus?
What do you do when your part doesn't interoperate with somebody
else's part? Indeed, how do you agree on what the requirements were,
so that you can tell management you're finished?
> When I explain things to people I watch their faces to see if they
> understand or not. If they do not understand I change the explanation
> so that they do understand.
When I think the documentation doesn't explain things well enough, I
bitch about it to the person responsible, and/or work it out on my own
and change the documentation.
> Also the *quality* of written matter in big projects often leaves much
> to be desired. How in 10 GBytes of framemaker documents can I find
> the relevant documentation. More often than not the written documents
> make no sense.
Yes! Exactly. And it's a vicious circle, isn't it? The
documentation sucks -> docs are useless -> nobody reads docs -> why
spend effort writing docs -> docs suck
Why on earth use Framemaker (we use Word, which is even worse)? Who
*makes* these decisions? Sigh.
There's quite a few companies around here that run fairly large
(non-software) projects. Say millions to a billion USD. AFAIK, they
have a clear project structure, strict requirements for documenting,
and they enforce it. And it works for them. I don't think software
is all that different.
(All this about a topic I know nothing about. Phew!)
-kzm
--
If I haven't seen further, it is by standing in the footprints of giants
Joe Armstrong
> >> If anything is a recipe for disaster it is quickly and chaotically
> >> communicating change requests over that wonderful corridor of
> >> yours...
>
> > I have worked in a number of different environments and feel quite
> > at home in the chaotic environment. Strangely these chaotic environments
> > produced high quality software - on time, on budget and with very few
> > errors
>
> > I have also worked in environments with strict change control -
> > here the results were not so good.
>
> So your corridors don't use change control, requirements,
> specifications or other documentation procedures? IMLE, these are
> really important for large projects.
>
Of course our corridor uses change control, requirements, specs you
name it. But we only use tools if they help us. I have seen many
tools that make the production of hight quality software more
difficult, not easier.
> > I have a feeling that any simple problem can be made arbitrary
> > difficult by imposing a suitably heavy administrative process around
> > the development.
>
> The problem IMHO is that engineers don't understand the point of
> development documentation, and thus ignore them. Or fill them with
> meaningless words, which is even worse.
>
> People need to understand that documentation has a purpose, and what
> that purpose is, and whom the intended audience is, before they write it.
>
> > Writing is fine - but it is not good for explaining things to people.
>
> So how do you get somebody to explain a piece of code, if that person
> quit the company four years ago? Or gets run over by a bus?
You shouldn't have to explain the code - that's what the spec is for.
Unfortunately much code gets re-written from scratch precisly because
the programmer who writes the stuff quits/dies - and this is even
when there is beautiful documentation!
>
> What do you do when your part doesn't interoperate with somebody
> else's part? Indeed, how do you agree on what the requirements were,
> so that you can tell management you're finished?
>
Well I'd like to use a type system and theorem prover - failing that
we have to resort to imprecise natural language and hope that both
sides of the interface correctly guess what the words in the document
mean.
I'd love to use Z or something but I've never seen a tool that's good
enough.
Unfortunately we have a problem, namely:
Natural langauge is good an explaining thing precisely because it
is sloppy and imprecise.
Maths/Types are bad at explaining things because they cannot talk
about imprecise things.
Specs. are wierd mixtures of precise and imprecise language which
may or may not capture the essence of a problem. Most real-world
problems that I have seen cannot be modelled in precise mathematical
terms.
> > When I explain things to people I watch their faces to see if they
> > understand or not. If they do not understand I change the explanation
> > so that they do understand.
>
> When I think the documentation doesn't explain things well enough, I
> bitch about it to the person responsible, and/or work it out on my own
> and change the documentation.
>
> > Also the *quality* of written matter in big projects often leaves much
> > to be desired. How in 10 GBytes of framemaker documents can I find
> > the relevant documentation. More often than not the written documents
> > make no sense.
>
> Yes! Exactly. And it's a vicious circle, isn't it? The
> documentation sucks -> docs are useless -> nobody reads docs -> why
> spend effort writing docs -> docs suck
>
> Why on earth use Framemaker (we use Word, which is even worse)? Who
> *makes* these decisions? Sigh.
Suits :-)
SGML is better (a bit anyway).
graham
> Ian Grant wrote:
>> I'm amazed to hear this. What about, say, the Linux kernel or any one of
>> dozens of other successful distributed development efforts? You must have
>> intended to qualify this claim and I'm genuinely intrigued to know how.
> I think the Linux kernel is atypical for the following reasons:
> (1) There wasn't any need for meetings or any form of co-ordination to specify
> the general outline, because Linux follows Unix fairly closely, and doesn't
> need any big new general design decisions.
I have to second this statement. In my experience there is a vast difference
between managing a project which is new stuff, and managing a project which
is essentially a reimplementation of old stuff. In the latter case the
project structure is mostly all there for you. In the former case you have
to find that structure.
graham
Mike Williams
|> I *strongly* believe that the only way to do large SW projects (Large
|> equals > 1 programmer) is to have all the programmers working in the
|> same corridor - even then it's very difficult and there will be a lot
|> of misunderstandings.
It depends entirely on what sort of software project we are talking
about. Projects in a new area, such as Bluetail's products, which
haven't been built before, must be done by programmers "working in the
same corridor". However many (most ?) software products are
derivatives or enhancements of existing products. Provided the the
existing products are well understood and have a reasonable
architecture, there is no reason why such projects cannot be
distributed to several sites. A good example is the AXE10 switching
system (the basis of 60% of the world GSM cellular systems and a
goodly proportion of the TDMA systems). Many projects in the AXE10
sphere are spread over the four corners of the globe. These projects
may not be the optimal way to develop software, but they deliver
working systems in keep the cash rolling in. As always there are
political, financial, practical reasons for doing things this way.
One the other hand spreading new product development this way has been
shown to be sub-optimal......
|> The key problem in software development is that different members of
|> the group do not really understand each other. Getting them to conduct
|> all conversations through e-mail is a recipe for disaster.
Sooner of later someone has to maintain the software produced by the
people from the "one corridor". It will almost certainly not be the
same people, they will have moved on to other innovative / amusing
projects. As the people in the same corridor won't have needed
specifications / or detailed descriptions of what they have done, the
poor maintainers won't have an easy time adapting the software to new
hardware or introducing new features, protocols etc. I'm not saying
that they should communicate by e-mail, but the fastest method to
develop software (i.e. the "one corridor" method), does require some
boring documentation if the product is to be maintained and it is
probably also necessary to introduce some of the people who are going
to keep with the product in the corridor as well. I.e. the fastest way
to design products probably does not result in very maintainable
software.
I think that the key to larger software development is to recognize
the what type of product the project addresses and to organize the
project accordingly.
/Mike
Scott Ribe
Christian Stapfer wrote:
> Of course, there just might exist exceptional
> individuals who keep learning and improving their
> skills even under *unfavourable* conditions. The
> example of Einstein working as a patent examiner
> comes to mind - but that was back in the 1910s,
> long before the apotheosis of Shareholder and
> Manager...
Uhm, this way off topic, but...
There are many examples in history of geniuses staying in menial jobs
while producing great works as a hobby. Could be that the menial job
allows one to put food on the table without taxing one's concentration,
leaving one's brain unencumbered for the creation process.
Yiorgos Adamopoulos
>I think the Linux kernel is atypical for the following reasons:
The Linux kernel is a bad example indeed because every new feature and
bugfix has to be accepted by Linux Torvalds before making it to the
standard kernel. However, good examples of where distributed development
worked successfully are, FreeBSD, NetBSD, OpenBSD and PostgreSQL.
There is a very nice page (can't find the URL right now) that describes
the difficulties involved for the PostgreSQL team when they took over
from the two Postgres95 maintainers.
-Y.
Christian Stapfer
Well, what I wanted to argue is that if people are
kept under *high pressure* in those menial jobs (since
nowadays, though perhaps not in Einstein's days,
Management is busy squeezing out profits as hard as
it can), they will simply *not* have enough energy
left to be as creative as Einstein was.
Another point, I believe, is this: It might be that
it was actually a *good* thing for Einstein *not* to be
"under one roof" with too many other physicists at the
time when he had to produce a profound *revolution* in
physics. The pressure to *conform* to 'current wisdom'
of those "under one roof" was lacking, which allowed
him to let his mind consider revolutionary alternatives
freely - you might want to consider how similar pressures
to conform might hurt the overall performance of a
software developer as well. (Gerald M. Weinberg tells
a story that seems to confirm such fears in the second
chapter of his "The Psychology of Computer Programming",
section "Specifications". I could supply many similar
stories from my own work experience.)
Christian