when does BDUF and waterfall work?
Gilligan
These methodologies should be tools to use when needed. Are there still
reasons to use these methodologies? Are agile approaches the new super
hammer?
Here is my take. We had reasons for big design up front. The reasons
varied from technical to territorial.
Reasons for BDUF: (No particular order)
1) Build times were submitted and it could be days before it was
loaded/compiled/ran, etc.
2) Debuggers where weak and it was more efficient to avoid bugs than
look for bugs.
3) Software management was emerging as a business and manage means
control.
4) Management flexed its new found power and said it will hold
developers accountable.
5) Developement needed a contract to be used in defense. (Development
new that all requirements are not known up front but management
believed that if more activities were performed then all of the
requirements could be discovered)
6) Software businesses emerge with many departments and with all of
these departments and people the business requires efficient use of the
human resources discounting the efficient development of code. This
means keeping marketing people busy at the expense of development. This
means keeping HR people busy at the expense of development. This means
keeping managers busy buzzy around at the expense of development. (Is
this enough to get a rise out of someone?)
In my humble opinion (IMHO) I feel that the reasons I list as (1) and
(2) are the most important. If there is something expensive to do (i.e.
build time) then do activities that are less expensive.
So in todays world when is BDUF useful. Suppose you will have only one
meeting with the customer. You had better get all you can get from that
meeting. Gather requirements. Create paper prototypes and present them
to entice further requirements from the customer. Because, what you
walk away with is what you have.
Can you think of another example?
Geoff
John Roth
news:1105547030....@c13g2000cwb.googlegroups.com...
Well, item 1 (long lead times to build and test) is a historical
fact, which thankfully is exactly that: historical. A queueing theory
analysis would show that it necessarilly forces big batches: doing
a lot of work to take advantage of the big batch nature of the
compile and test system.
Item 2 turns out to be a red herring. One of the most consistent,
and most surprising, things to come out of experiance in using
Test Driven Development is that people use the debuggers less,
to the point where they frequently forget how. A frequent need to
use a debugger is a sign of poor development practices. (Smalltalk
is somewhat of an anomaly since you can change code and restart
from the debugger.)
3 through 6 are all organizational issues. Many of them are
forced by the nature of big, hierarchical organizations; one
doesn't have to work that way, nor do organizations have
to work that way.
What's going on is a simple (lack of) recognition that when
you remove one bottleneck (long lead time for access to
compile and test resources), the bottleneck moves somewhere
else, and sometimes it makes sense to reorganize the process
to take advantage of the new facilities.
John Roth
>
> Geoff
>
Gilligan
In TDD you find that "people" use debuggers less. Please expound.
Which people?
How do you quantify less?
I find that when it is necessary to show behavior to another developer
(we are going to pair on something) that running the tests and stepping
through the code is a great way. The debugger is essential and used a
lot.
How did you mean your statement?
Thanks,
Geoff
John Roth
It's a general observation of people who use TDD a lot.
The don't find a real need to use the debugger because
problems show up when you run the test suite, which
you do after every edit that you change something.
Since it ran correctly before the last change, it's the last
change that caused the failing tests. It's usually quite
obvious as to what happened, and if it isn't it's usually
faster to hit undo and do something different.
If you find that the debugger helps you, fine. There are
people that like it.
John Roth
>
> Thanks,
> Geoff
>
Ron Jeffries
wrote:
>Thanks for your comments.
>
>In TDD you find that "people" use debuggers less. Please expound.
>Which people?
I'm not the OP, but "people" to me means some unidentified subset of
the people I talk to who have taken up TDD. A very common statement is
that they no longer need to do debugging.
>How do you quantify less?
I have programmed in Ruby for several years now, off and on, and I
don't even know how to use the debugger.
In Smalltalk I use the debugger as a code entry tool, because it will
automatically put you in the write place to type in the method or make
the change, but I use it almost never to "debug", i.e. to look at
objects, much less step.
>
>I find that when it is necessary to show behavior to another developer
>(we are going to pair on something) that running the tests and stepping
>through the code is a great way. The debugger is essential and used a
>lot.
If code can only be understood through stepping, I would be concerned
about its clarity and the quality of its design. I'm not saying that
it is automatically bad, but I'd be looking pretty hard at trying to
make it more clear. We should not have to step code to understand it,
in my opinion.
Regards,
--
Ron Jeffries
www.XProgramming.com
I'm giving the best advice I have. You get to decide if it's true for you.
Ron Jeffries
wrote:
>In Smalltalk I use the debugger as a code entry tool, because it will
>automatically put you in the write place to type in the method or make
>the change, but I use it almost never to "debug", i.e. to look at
>objects, much less step.
I guess I didn't right the write thing up there. :)
Gilligan
about its clarity and the quality of its design. I'm not saying that
it is automatically bad, but I'd be looking pretty hard at trying to
make it more clear. We should not have to step code to understand it,
in my opinion.
I will focus on the word "only" in your response. It is not the only
way. But the debugger is a great tool. It keeps context for you so that
the humans are not burdened. It shows all of the state for you so that
you don't have to keep that in your head. It follows the paths that you
want (either the natural code flow or move the PC) so it is like a
"reader" in the old reviews we used to do. You can stop during the
"tutorial" and go to lunch and it is easy to look at the state and
remember where you were. It is a tool.
When you read code you read it one step at a time. Don't you? Depending
on your current role in development you may have to read someones code.
To take your comment to the extreme it would be like saying, "We should
not have to read code to understand it." That is fine, the unit tests
"understand" the code and whats to understand if the code works? Lots
of things. You may be refactoring a piece that you are not aware of.
You have to read the code to understand it, don't you? If the original
programmer has changed jobs (yet again) you can not ask for human
explination and if he was there would you trust his memory or would you
run his tests and read/step through them to get the picture?
How do you do it if my approach isn't write (he he pun from your
followup)?
Geoff
aka Gilligan
aka Maverick
aka Trouble!
Shane Mingins
news:1105639619.0...@z14g2000cwz.googlegroups.com...
> To take your comment to the extreme it would be like saying, "We should
> not have to read code to understand it."
Actually the point I get from Ron's comment is that "We should understand
the code by reading it".
Good examples that I have seen of this recently are in the Refactoring to
Patterns book by Joshua Kerievsky.
Cheers Shane
Gilligan
me to do the things I do.
A couple of which I haven't mentioned before is that I am slightly
dislexic (sp?). If I look at number quickly I will transpose some of
them. Because of this I try to be ultra focused. I guess I have learned
to hide my limp.
Also, I do too many things and the contexts get all jumbled up.
Maybe I am a handicapped programmer!
Robert C. Martin
wrote:
>For years we used Waterfall methodologies and built amazing systems.
Are you speaking of yourself, or are you speaking of the industry.
The statistics for the industries use of waterfall are appalling.
Waterfall project failed by 2:1 or more. Craig Larman's wrote a book
on this topic recently. You might want to glance through the 50 pages
of evidence about the fiasco that Waterfall really is.
Agile and Iterative Development: A Manager's Guide
http://www.amazon.com/exec/obidos/ASIN/0131111558/objectmentorinc
>These methodologies should be tools to use when needed. Are there still
>reasons to use these methodologies? Are agile approaches the new super
>hammer?
Agile methods are the *old* super hammer. They predate waterfall
significantly -- though they've only been given the name "agile"
recently, and their practices have only been refined in the last
decade.
>So in todays world when is BDUF useful.
A formalized design phase with formalized artifacts is never useful
IMHO.
>Suppose you will have only one meeting with the customer.
You will fail. So make sure the contract is time and materials.
>You had better get all you can get from that
>meeting. Gather requirements. Create paper prototypes and present them
>to entice further requirements from the customer. Because, what you
>walk away with is what you have.
>Can you think of another example?
And even then I would use an agile process. I would have someone
internal represent the customer, spending the vast bulk of their time
trying to understand the customer's needs and the features that will
address them.
-----
Robert C. Martin (Uncle Bob) | email: uncl...@objectmentor.com
Object Mentor Inc. | blog: www.butunclebob.com
The Agile Transition Experts | web: www.objectmentor.com
800-338-6716
"The aim of science is not to open the door to infinite wisdom,
but to set a limit to infinite error."
-- Bertolt Brecht, Life of Galileo
ig...@yahoo.com
> Waterfall project failed by 2:1 or more.
Shouldn't that be
"Yes, it's possible to completely screw up Waterfall projects"?
http://groups-beta.google.com/group/comp.software.extreme-programming/msg/9277a59c0e051e3a
Ron Jeffries
>> The statistics for the industries use of waterfall are appalling.
>> Waterfall project failed by 2:1 or more.
>
>Shouldn't that be
>"Yes, it's possible to completely screw up Waterfall projects"?
I'm not sure what Bob believes, but I believe that Waterfall, and
waterfall-style projects are in fact more likely to be screwed up than
are projects of corresponding size done in an iterative and
incremental fashion.
Craig Larman's book, chapter 6 IIRC, is full of studies, some quite
large (tens of thousands of projects) bearing on this concern.
Regards,
John Roth
<ig...@yahoo.com> wrote in message
news:1105928941.4...@z14g2000cwz.googlegroups.com...
>> The statistics for the industries use of waterfall are appalling.
>> Waterfall project failed by 2:1 or more.
>
> Shouldn't that be
> "Yes, it's possible to completely screw up Waterfall projects"?
The point isn't that it's _possible_. The point is that
it's done with disgusting regularity. The only strategy
that works consistently is incremental development
with installs at intervals of no longer than three months.
The fact is, it's possible to screw up anything.
With short delivery cycles, it's impossible to
hide the screwup for too long.
You can run a mini-waterfall inside of the three
month time boxes. It isn't the best way to go,
but it will frequently work.
John Roth
David Wolff
With stats like that, I think it's more accurate to say
"Yes, it's *usual* to completely screw up Waterfall projects."
Thanks --
David
(Remove "xx" to reply.)
ig...@yahoo.com
> "Yes, it's *usual* to completely screw up Waterfall projects."
ig...@yahoo.com
> month time boxes. It isn't the best way to go,
> but it will frequently work.
At which point, we just seem to be saying
http://history1900s.about.com/library/graphics/wwiip247.jpg
Robert C. Martin
One difference is that the guy in the quoted article was describing an
environment which he called XP, but certainly was not. On the other
hand, one of the more compelling studies in Larman's book shows that
the more rigorously you follow waterfall, the worse your results are.
So, while it is possible to screw up both waterfall and XP, it appears
that you screw up XP by not doing it, whereas you screw up waterfall
by doing it.
John Roth
"Robert C. Martin" <uncl...@objectmentor.com> wrote in message
news:cq3ou01l7j1jms7fb...@4ax.com...
> On 16 Jan 2005 18:29:01 -0800, ig...@yahoo.com wrote:
>
>>> The statistics for the industries use of waterfall are appalling.
>>> Waterfall project failed by 2:1 or more.
>>
>>Shouldn't that be
>>"Yes, it's possible to completely screw up Waterfall projects"?
>>
>>http://groups-beta.google.com/group/comp.software.extreme-programming/msg/9277a59c0e051e3a
>
> One difference is that the guy in the quoted article was describing an
> environment which he called XP, but certainly was not. On the other
> hand, one of the more compelling studies in Larman's book shows that
> the more rigorously you follow waterfall, the worse your results are.
>
> So, while it is possible to screw up both waterfall and XP, it appears
> that you screw up XP by not doing it, whereas you screw up waterfall
> by doing it.
While on the gripping hand, there are several different
methodologies that are called waterfall, and which have
very different success rates. A strict phasist waterfall
usually doesn't work for anything larger than a single
developer, half year project. The version that's embedded
in the SEI's TSP (Team Software Process) will work with
short delivery cycles as long as the project manager doesn't
lie to himself about how long the big bang integration at the
end will take.
That doesn't mean it's the best methodology - just that
it can be made to work with an experianced project
manager (see the Chaos Reports for where "Experianced
Project Manager" falls on the success factors ladder.)
John Roth
Isaac Gouy
> an environment which he called XP, but certainly was not. On the
> other hand,
> one of the more compelling studies in Larman's book shows that the
> more rigorously you follow waterfall, the worse your results are.
Which study?
> So, while it is possible to screw up both waterfall and XP, it
> appears that you screw up XP by not doing it, whereas you screw
> up waterfall by doing it.
We can say we're doing 'XP' when we're not; and we can say we're doing
'Waterfall' when we're not.
We can do 'XP' when it isn't appropriate; and we can do 'Waterfall'
when it isn't appropriate.
The author of "Implementing the IEEE Software Engineering Standards"
starts off enthused:
"Although widely disparaged as inefficient, the waterfall model, the
oldest, useful, lifecycle model still has many advantages... For
projects that lend themselves to a sequential scheduling approach, a
simple waterfall model is often the most efficient. Only when the
project risks are substantial enough to require more flexibility in
task scheduling does the waterfall model compare unfavourably with more
sophisticated models... A strict implementation of the sequencing
phases in a waterfall model is almost never used."
When does 'Waterfall' work?
"The communications protocols to be handled were effectively cast in
concrete by manufacturers and international standards, the loadings
were not expected to be high, the switching was straighforward, and
only the failure conditions gave us any real concern. The development
proceeded absolutely classically: a system definition agreed with the
client; a design completed before coding started; coding completed
before testing and integration started. Since the risks were low we
could plan and estimate with confidence and use a simple model for the
project... when I look back, I realise that it was successful simply
because the problem and solution domains were so well understood by the
team I had working for me! With a different team the situation could
have been quite different, and, with the new risks and uncertainties, I
would have been wise to have chosen a model that allowed more risk
reduction en route."
Strategies for Software Engineering: The Management of Risk and Quality.
Isaac Gouy
> I'm not sure what Bob believes, but I believe that Waterfall, and
> waterfall-style projects are in fact more likely to be screwed up
> than are projects of corresponding size done in an iterative and
> incremental fashion.
John Roth
"Isaac Gouy" <ig...@yahoo.com> wrote in message
news:1106033113.5...@z14g2000cwz.googlegroups.com...
>
> The author of "Implementing the IEEE Software Engineering Standards"
> starts off enthused:
[...]
> When does 'Waterfall' work?
> "The communications protocols to be handled were effectively cast in
> concrete by manufacturers and international standards, the loadings
> were not expected to be high, the switching was straighforward, and
> only the failure conditions gave us any real concern. The development
> proceeded absolutely classically: a system definition agreed with the
> client; a design completed before coding started; coding completed
> before testing and integration started. Since the risks were low we
> could plan and estimate with confidence and use a simple model for the
> project... when I look back, I realise that it was successful simply
> because the problem and solution domains were so well understood by the
> team I had working for me! With a different team the situation could
> have been quite different, and, with the new risks and uncertainties, I
> would have been wise to have chosen a model that allowed more risk
> reduction en route."
> Strategies for Software Engineering: The Management of Risk and Quality.
I have relatively little difficulty with the notion of doing the
requirements
work up front in this situation. I suspect I'd still go with TDD rather than
separate design, code, test and integrate steps, though.
John Roth
>
Gilligan
that many great systems were built using phasic approaches and I wanted
to know what we can use from that experience.
Many have said what systems? Well, I can list the ones I think are
amazing and then everyone will challenge this because it is subjective.
Isaac Gouy
> I have relatively little difficulty with the notion of doing the
> requirements work up front in this situation. I suspect I'd still
> go with TDD rather than separate design, code, test and integrate
> steps, though.
With the best intention, imo a more interesting answer would say why
you would 'go with' some particular technique - habit? confidence in
the technique? prejudice? easier than trying to remember what other
techniques could be used?
John Roth
I normally assume that people on this ng understand (at least
intellectually) the difference between the fine grained integration
of design, coding and test in TDD, and the course grained
non-integration implied in doing all of your design, then all of your
coding, then all of your testing.
I know of no case where doing a complete design, without
testing it as you go along, is superior to testing each element
as you create it by writing the tests and code. I likewise know
of no case where writing a lot of code and then testing it
afterwards is superior to specifying exactly what each piece
of code is to do before hand by writing a test, then writing the
code, then verifying that it does what you wanted it to do.
Even people who violently object to aspects of XP are
taking up Test Driven Development. Why? It works, and
it works well.
John Roth
>
Isaac Gouy
So no newbies expected here? ;-)
> I know of no case where doing a complete design, without
> testing it as you go along, is superior to testing each element
> as you create it by writing the tests and code. I likewise know
> of no case where writing a lot of code and then testing it
> afterwards is superior to specifying exactly what each piece
> of code is to do before hand by writing a test, then writing the
> code, then verifying that it does what you wanted it to do.
Seems that you've linked an objective with a particular way of
achieving that objective. Is code and test always the best verification
technique?
> Even people who violently object to aspects of XP are
> taking up Test Driven Development. Why? It works, and
> it works well.
Fashionistas unite!
Robert C. Martin
Good choice!
Robert C. Martin
<newsg...@jhrothjr.com> wrote:
>While on the gripping hand,
(good book!)
Robert C. Martin
>> One difference is that the guy in the quoted article was describing
>> an environment which he called XP, but certainly was not. On the
>> other hand,
>
>> one of the more compelling studies in Larman's book shows that the
>> more rigorously you follow waterfall, the worse your results are.
>
>Which study?
It's a study that he calls [Solon02]. Specifically Larman says that
it was a study of 43,000 projects that found:
• Higher productivity with iterative, rather than
waterfall methods, and the more “rigorously”
the waterfall was followed, the lower the
productivity and success rates.
John Roth
>> I normally assume that people on this ng understand...
> So no newbies expected here? ;-)
You've been on this ng for a while.
>
>> I know of no case where doing a complete design, without
>> testing it as you go along, is superior to testing each element
>> as you create it by writing the tests and code. I likewise know
>> of no case where writing a lot of code and then testing it
>> afterwards is superior to specifying exactly what each piece
>> of code is to do before hand by writing a test, then writing the
>> code, then verifying that it does what you wanted it to do.
>
> Seems that you've linked an objective with a particular way of
> achieving that objective. Is code and test always the best verification
> technique?
You're weasling out by not quoting the context I was replying
to. You can prove anything that way, and make any comment
look stupid by doing it.
I have no idea what the "best" verification method is. Define
what "best" means, in what context and what measures you
are using to define "best", and I might be able to render an
opinion.
>> Even people who violently object to aspects of XP are
>> taking up Test Driven Development. Why? It works, and
>> it works well.
>
> Fashionistas unite!
Did you get up on the wrong side of the bed this morning?
John Roth
>
Isaac Gouy
> Good choice!
It certainly can be.
Isaac Gouy
Isaac Gouy
> >> testing it as you go along, is superior to testing each element
> >> as you create it by writing the tests and code. I likewise know
> >> of no case where writing a lot of code and then testing it
> >> afterwards is superior to specifying exactly what each piece
> >> of code is to do before hand by writing a test, then writing the
> >> code, then verifying that it does what you wanted it to do.
> >
> > Seems that you've linked an objective with a particular way of
> > achieving that objective. Is code and test always the best
> > verification technique?
>
> You're weasling out by not quoting the context I was replying
> to. You can prove anything that way, and make any comment
> look stupid by doing it.
Hadn't realized there was something I needed to weasel out of, what is
it?
(Been using threaded news-readers for so long that it's hard to
remember some people might still be getting one post at a time.)
Surprised you consider what you wrote stupid - I'm just trying to
understand what you think.
> I have o idea what the "best" verification method is. Define
> what "best" means, in what context and what measures you
> are using to define "best", and I might be able to render an
> opinion.
Well, I had no idea what you meant by "superior" but it didn't seem
worth squabbling about. I'll rephrase- "Is code and test always a
superior verification technique?".
> >> Even people who violently object to aspects of XP are
> >> taking up Test Driven Development. Why? It works, and
> >> it works well.
> > Fashionistas unite!
> Did you get up on the wrong side of the bed this morning?
imo Fashion is an important force in all manner of things, including
software development.
Scott Kinney
actually
having to, say, read it.news:1106112316....@f14g2000cwb.googlegroups.com...
Robert C. Martin
>> It's a study that he calls [Solon02]
>Isn't the full reference given in the bibliography?
Probably, but both my copies of Larman's book have been borrowed and
not returned. I drew my citing from personal notes.
Isaac Gouy
> It's a study that he calls [Solon02]. Specifically Larman says that
> it was a study of 43,000 projects that found:
>
> · Higher productivity with iterative, rather than
> waterfall methods, and the more "rigorously"
> the waterfall was followed, the lower the
> productivity and success rates.
1) There were a total of 43,700 projects (from 1991 onwards) in the
Gartner database. The loose/moderate/rigorous comparison only used data
from "the last two years" - not the 11 years of data in the database,
not 43,700 projects.
The article doesn't say how many projects were included in this
comparison - we aren't told the sample size.
2) The article never mentions "success rates"
3) "The benchmark also asks respondents to identify their generic life
cycle as one of waterfall or prototyping,..."
The article doesn't say how many projects were identified as waterfall
or prototyping - 1 prototype vs 10 waterfall? vica versa? we don't
know.
4) "... why such a lower productivity at moderate or rigorous level,
when compared to the loose level of rigor... they (sic different study
of a COBOL MRP system) found that for the full product life cycle, the
impact of improved quality outweighed the decreased development
productivity, because of its positive effect on the long term
maintenance work... we conjecture that the data in the Gartner database
also portrays this same reduction in productivity during development,
as more process rigor is applied."
I don't have Larman's book so it would be foolish for me to make any
comments about it. Let's just take what you noted at face value, then
we have:
- basic factual error,
the productivity information was not based on 43,000 projects
- basic factual error,
"success rates" are not mentioned
- fudge in the study
what are the sample sizes?
were the respondents told how to interpret prototype/waterfall?
- different interpretations of the data
the original authors chose to explain the productivity decrease
by a conjectured quality increase
This is why I ask for the original source information.
Robert Solon Jr., and Joyce Stratz 2002
Benchmarking the ROI for Software Process Improvement (SPI),
Software Tech News Volume 5, Number 4
http://www.softwaretechnews.com/stn5-4/benchmarking.html
1.8MB http://www.softwaretechnews.com/stn5-4/stn5-4.pdf
Scott Kinney
I haven't read the Solon paper, but I had similar complaints comparing the
Larman's representation of the Standish Group's Chaos report with the actual
report itself.
"Isaac Gouy" <ig...@yahoo.com> wrote in message
news:1106175691....@f14g2000cwb.googlegroups.com...
Scott Kinney
Barry Boehm's book "Balancing Agility and Discipline" approaches it as a
'custom mix'. By examining the specific attributes of the project,
technologies,
team, organization, and so on; the project manager and team draw specific
approaches
from different methodologies for different parts of the project.
"Gilligan" <geoffrey...@gmail.com> wrote in message
news:1106069387....@f14g2000cwb.googlegroups.com...
Isaac Gouy
>From page 76
"A study [Solon02] against a sample set (43,700 projects) showed the
following productivity differences between IID and waterfall:
Rigorous IID or Evolutionary Prototyping 570 FP
Rigorous Waterfall 480 FP
Interestingly, the same study showed that among the waterfall projects,
those that applied it only 'loosely' were significantly more productive
than those that applied it 'rigorously', indicating the negative effect
it has on productivity."
So there are a couple of problems:
- the productivity differences are from the last two years; but the
43,700 projects are from 1991-2002
- the original authors chose to explain the productivity decrease by a
conjectured quality increase
(Didn't see any mention of "success rates".)
Robert C. Martin
question is on page 6. Larman's quote: <"Interestingly, the same
study [Solon02] showed that among the waterfall projects, those that
applied it only "loosely" were significantly *more* productive than
those that applied it "rigorously," indicating the negative effect
that it has on productivity.>
I think you are correct in assuming that Larman missed the fact that
only the last two years of the study were used, and that all 43,700
projects were therefore not involved in the study.
Larman did not claim lower "success" rates in his book. That claim
was made in my notes of a talk he gave. The words in my notes are:
"Higher productivity with iterative, rather than waterfall methods,
and the more "rigorously" the waterfall was followed, the lower the
productivity and success rates."
It is not unreasonable to presume that cutting the productivity in
half, as the study shows that rigorously following waterfall does,
implies less success. So, although my notes of his talk are a
stretch, they are not unreasonable.
The study did say that in one MRP system (comprised of 30 products
that were not part of the study), rigor led to greater quality, which
made up for the lower productivity. However, it did not verify that
there was higher quality in the waterfall projects in the study. In
fact it's not clear that this other system was developed with
waterfall.
The other interesting fact in the study is that rigor in prototyping
methods *increases* productivity almost by a factor of two. Moreover
the productivity of the rigorous prototyping was higher than the
productivity of the "loose" waterfall.
If their implication that rigor leads to quality is true of
"prototyping" methods (and it certainly is in agile methods employing
TDD) then one can use that extra quality as a bonus above and beyond
the increased productivity, rather than to make up for the lower
productivity of waterfall.
So, I infer that the more rigorously you do agile, the better it
works; whereas the more rigorously you do waterfall, the worse it
works.
On 19 Jan 2005 15:01:31 -0800, "Isaac Gouy" <ig...@yahoo.com> wrote:
>I don't have Larman's book so it would be foolish for me to make any
>comments about it. Let's just take what you noted at face value, then
>we have:
>
>- basic factual error,
>the productivity information was not based on 43,000 projects
>
>- basic factual error,
>"success rates" are not mentioned
>
>- fudge in the study
>what are the sample sizes?
>were the respondents told how to interpret prototype/waterfall?
>
>- different interpretations of the data
>the original authors chose to explain the productivity decrease
>by a conjectured quality increase
>
>This is why I ask for the original source information.
>
>
>Robert Solon Jr., and Joyce Stratz 2002
>Benchmarking the ROI for Software Process Improvement (SPI),
>Software Tech News Volume 5, Number 4
>
>http://www.softwaretechnews.com/stn5-4/benchmarking.html
>1.8MB http://www.softwaretechnews.com/stn5-4/stn5-4.pdf
-----
Isaac Gouy
> only the last two years of the study were used, and that all 43,700
> projects were therefore not involved in the study.
Yeah, inspection, inspection, inspection.
> Larman did not claim lower "success" rates in his book. That claim
> was made in my notes of a talk he gave. The words in my notes are:
> "Higher productivity with iterative, rather than waterfall methods,
> and the more "rigorously" the waterfall was followed, the lower the
> productivity and success rates."
>
> It is not unreasonable to presume that cutting the productivity in
> half, as the study shows that rigorously following waterfall does,
> implies less success. So, although my notes of his talk are a
> stretch, they are not unreasonable.
The comment on "success rates" is unconnected with the referenced
study. (Probably just one of those simple mistakes that happen when
people talk.)
> did not verify that there was higher quality in the
> waterfall projects
Yeah, it's pure speculation by the authors - no data on quality in the
Gartner database.
> The other interesting fact in the study is ...
imo the interesting fact about the study was that anyone cited it!
High school kids do better work!
There's little point speculating about the results, when we don't know
the most basic information. For example, maybe the average "prototype"
project was 10,000x smaller than the average "waterfall" project.
Isaac Gouy
> large (tens of thousands of projects) bearing on this concern.
Have you taken a look at the studies?
iirc Larman's book quotes this from "IT projects: sink or swim?"
"This suggests that the management of expectations is poor and the
approach of full requirements definition followed by a long gap before
those requirements are delivered is no longer appropriate."
After reading and re-reading 2 different versions of the report
http://www.bcs.org/BCS/Products/Publications/JournalsAndMagazines/ComputerBulletin/OnlineArchive/jan00/professionalpractice.htm
http://www.bcs.org/review/2001/articles/itservices/projects.htm
I have been unable to find any statement of how the 1,027 projects were
being managed - so I'm perplexed that the author feels able to talk
about a gap between requirements definition and delivery. (afaict they
could all have been using XP.)
John Roth
news:tud0v01cjrqt00dp8...@4ax.com...
>I have my copy of Larman's book in front of me now. The citing in
> question is on page 6. Larman's quote: <"Interestingly, the same
> study [Solon02] showed that among the waterfall projects, those that
> applied it only "loosely" were significantly *more* productive than
> those that applied it "rigorously," indicating the negative effect
> that it has on productivity.>
The biggest problem I have with this entire set of speculations
is that the processes are too loosely defined. There are at least
two very different processes that go under the name of Waterfall,
for example.
The strict phasist waterfall is well known to deteriorate the
larger the project: it can work for projects with a single
developer and no more than 6 months, and gets progressively
worse after that.
However, the gold standard doesn't do that. The process taught
in the SEI's PSP and TSP is pretty standard "software engineering"
process, which is also called waterfall. There are well known problems
with it, including spending too much time on requirements, getting
bogged down in change requests and
a tendency to enter "integration hell" due to fuzzing testing during
the actual development phase, but a good project manager (see
the Chaos reports) can avoid much of that.
"Loosely" can be interpreted as saying that the standard
"software engineering" process was applied instead of phasist
waterfall.
What they mean by "prototyping" processes is likewise insufficiently
defined to reach any kind of a conclusion.
The only thing one can do with this is to say "insufficient data",
and ignore it. The Chaos reports give a much clearer picture.
As far as I'm concerned, I agree completely with Alistair
Cockburn (crystal Clear, p. 71) that the single best indicator
of success is the delivery of tested, running software at
intervals of no more than 3 months. (The second is
reflective improvement.)
A close look at the data in the Chaos reports sustains
this view. It's my personal experiance on an aparently
doomed project (the executive sponsor had too much
political captial tied up in it to kill it) that going to a
"deliver tested, running software that meets the customer's
priorities every 3 months, come heck or high water"
regime pulled it out of the fire. It finished when it was
needed.
It's also the party line that's being repeated by more
and more of the people in the field. It doesn't seem
to matter what the process is, either. Even a monolithic
waterfall can succeed if it has to deliver in 3 month
increments.
John Roth
Isaac Gouy
> The only thing one can do with this is to say "insufficient data",
> and ignore it.
Yes.
(Mr Martin, the results of the study may have been to your liking, but
let's hope this was not "one of the more compelling studies in Larman's
book".)
-snip-
> It doesn't seem to matter what the process is, either. Even a
> monolithic waterfall can succeed if it has to deliver in 3 month
> increments.
That level of simplicity would be an interesting starting point.
John Roth
I'd suggest you read "Crystal Clear". It'll give you some
perspective on the entire issue.
John Roth
>
Robert C. Martin
Here is the context of your quote from the study:
"While scope management was some way ahead of the other factors, all
options scored, showing there is no single issue which repeatedly
causes failure. Managers were also asked at which stage failure could
occur. Figure 2 shows the results.
"Again, all stages were selected, but failure at the requirements
definition stage led the way by some distance, with implementation and
acceptance testing also scoring highly. The first stage deals with the
statement of expectations and the next two cover the delivery of those
expectations: often the first time the clients see the results.
"This suggests that the management of expectations is poor and the
approach of full requirements definition followed by a long gap before
those requirements are delivered is no longer appropriate."
The concept of a "requirements definition stage" is a waterfall
concept. Agile Methods do not have such a stage. So it seems
unlikely that any of the respondents who selected "requirements
definition stage" as a cause of failure would have been working in XP,
or an Agile Method.
Robert C. Martin
>imo the interesting fact about the study was that anyone cited it!
>High school kids do better work!
>
>There's little point speculating about the results, when we don't know
>the most basic information. For example, maybe the average "prototype"
>project was 10,000x smaller than the average "waterfall" project.
Studies are studies. They attempt to abstract out some information in
order to amplify a signal in the noise. Every study is incomplete,
and high school kids can always do better (just ask them).
When a study suggests something we don't like, we find fault with it.
When a study suggests something we do like, we cite it. That's just
human nature.
Sometimes, though rarely, we actually consider the signal that's
trying to peek out from the noise.
In this case there seems to be a signal that rigor in a prototyping
method has a positive effect on productivity, whereas rigor in a
waterfall method has a negative effect on productivity.
We can argue about whether that signal is produced from 43,700
projects or not. We can argue about the size of the projects. We can
argue about high school kids. But the signal is still there.
Robert C. Martin
<newsg...@jhrothjr.com> wrote:
>It's also the party line that's being repeated by more
>and more of the people in the field. It doesn't seem
>to matter what the process is, either. Even a monolithic
>waterfall can succeed if it has to deliver in 3 month
>increments.
My definition of waterfall does not include the concept of increments.
Once there are increments, you aren't doing waterfall anymore, you are
doing incremental development.
Robert C. Martin
>-snip-
>> The only thing one can do with this is to say "insufficient data",
>> and ignore it.
Data is always insufficient. There seems to be a signal in the noise.
>
>Yes.
>
>(Mr Martin, the results of the study may have been to your liking, but
>let's hope this was not "one of the more compelling studies in Larman's
>book".)
Sorry to dash your hopes, but I find the study to be compelling. It
shows evidence of a bizarre negative correlation between rigorous
waterfall and productivity. It also shows evidence of a positive
correlation between rigorous prototyping and productivity. That's
*interesting*.
Steve Jorgensen
>As I talk to people I think I have found some of the things that cause
>me to do the things I do.
>
>A couple of which I haven't mentioned before is that I am slightly
>dislexic (sp?). If I look at number quickly I will transpose some of
>them. Because of this I try to be ultra focused. I guess I have learned
>to hide my limp.
>
>Also, I do too many things and the contexts get all jumbled up.
>Maybe I am a handicapped programmer!
All of us, if we're honest, are handicapped programmers in some such way.
Using TDD is a fine way to compensate for many of our handicaps, because
problems are detected and fixed almost as they're written, and the continual
refactoring keeps code executions paths clear and simple enough that they can
be understood by looking at them (for the most part - when we're doing TDD
right). I now find that I only have a use for the debugger when I've been
failing to refactor well, and am now in the process of getting rid of a tangle
I should have been avoiding along the way.
Isaac Gouy
> >but let's hope this was not "one of the more compelling studies
> >in Larman's book".)
>
> Sorry to dash your hopes, but I find the study to be compelling. It
> shows evidence of a bizarre negative correlation between rigorous
> waterfall and productivity. It also shows evidence of a positive
> correlation between rigorous prototyping and productivity. That's
> *interesting*.
You're right - I am hopeful. Hopeful that the other studies have more
substance; and hopeful that most people can look at evidence, notice
the unknown variables, and realize how much that destroys the
credibility of any conclusions we may wish to make.
Isaac Gouy
And drivel is drivel.
-snip-
> In this case there seems to be a signal that rigor in a prototyping
> method has a positive effect on productivity, whereas rigor in a
> waterfall method has a negative effect on productivity.
>
> We can argue about whether that signal is produced from 43,700
> projects or not. We can argue about the size of the projects.
> We can argue about high school kids. But the signal is still there.
And "the signal" may be related to extreme differences in "the size of
the projects" and not related to the methods used on the projects.
(The sad thing is that the information for how many projects were in
that 2 year sample was certainly available, and the FP size of the
projects was probably available as well.)
Isaac Gouy
John Roth has made me hypersensitive ;-)
> Managers were also asked at which stage failure could occur...
-snip-
> The concept of a "requirements definition stage" is a waterfall
> concept. Agile Methods do not have such a stage. So it seems
> unlikely that any of the respondents who selected "requirements
> definition stage" as a cause of failure would have been working
> in XP, or an Agile Method.
Let's say you're managing a project using XP, you feel that getting the
requirements is a big issue (that is one of the reasons you're using
XP), so which box do you tick when "asked at which stage failure could
occur"?
John Roth
If you're doing XP, getting the requirements isn't a big
issue.
John Roth
>
John Roth
> On Thu, 20 Jan 2005 22:03:18 -0600, "John Roth"
> <newsg...@jhrothjr.com> wrote:
>
>>It's also the party line that's being repeated by more
>>and more of the people in the field. It doesn't seem
>>to matter what the process is, either. Even a monolithic
>>waterfall can succeed if it has to deliver in 3 month
>>increments.
>
> My definition of waterfall does not include the concept of increments.
> Once there are increments, you aren't doing waterfall anymore, you are
> doing incremental development.
It's what you do inside the increments that's interesting.
It's quite possible to do a monolithic waterfall inside
a 3 month increment. You collect requirements for 18
days, you design for 18 days, you code for 8 days,
you test for so many days, then you integrate and so
forth.
Incremental development gives you a short enough
time frame for a reasonable likelihood of success.
The project I mentioned in another post got pulled
out of the fire when the last manager insisted on
doing incremental development: deliver every three
months. We still did requirements, then design, then
split up to code and test, and then did integration
at the end. It worked. That's not to say it couldn't
have worked better: the PM in charge of the UI
was always complaing about coordination with the
batch process not happening right.
The confusion seems to be equating "Waterfall"
with "excessively long lead time to delivery," or
"one delivery per project." Maybe that's the way
other people use it, but I don't see it that way.
John Roth
Isaac Gouy
Ilja Preuß
> You're right - I am hopeful. Hopeful that the other studies have more
> substance; and hopeful that most people can look at evidence, notice
> the unknown variables, and realize how much that destroys the
> credibility of any conclusions we may wish to make.
They cannot be waterproof. There are always unknown variables, with every
study. If everything else fails, attribute the difference to personal
factors. Using that tactic, you can nullify *any* conclusion.
Cheers, Ilja
Isaac Gouy
Ilja, have you looked at the study?
Laurent Bossavit
> You're right - I am hopeful. Hopeful that the other studies have more
> substance; and hopeful that most people can look at evidence, notice
> the unknown variables, and realize how much that destroys the
> credibility of any conclusions we may wish to make.
I fear that "most people" are not as diligent as you are in going back
to primary sources. (Or close approximations thereof.)
Not many seem to have read the Royce paper either; the one that says
that the strictly sequential process "is risky and invites failure".
Many people, over the years, have apparently drawn conclusions from
Royce, from DoD standards, and so on, which never had much credibility
to begin with. We're all asking "when does method X work", and can't
seem to agree on either the nature of any X or the meaning of "work".
The "quantitative" studies mostly seem to have so many unknow variables
that they add to the mess, rather than offer hope of sorting it out.
What's to be done about all that ?
Laurent
Ilja Preuß
No, I haven't.
And just to be clear: I don't advice to fully trust any study.
I actually don't think that anyone uses studies as there primary source for
building opinions. At least I've never seen someone changing his opinion
based on one.
Cheers, Ilja
Ilja Preuß
> Many people, over the years, have apparently drawn conclusions from
> Royce, from DoD standards, and so on, which never had much credibility
> to begin with.
Yes, but if they had understood those papers right, would that have changed
what they had done? I somehow strongly doubt that...
Cheers, Ilja
Isaac Gouy
> > have more substance; and hopeful that most people can look
> > at evidence, notice the unknown variables, and realize how much
> > that destroys the credibility of any conclusions we may wish
> > to make.
>
> I fear that "most people" are not as diligent as you are in going
> back to primary sources.
I was taught to check primary sources. (Authors are sometimes too
selective and sometimes embellish, and sometimes simply make mistakes.)
-snip-
> Many people, over the years, have apparently drawn conclusions from
> Royce, from DoD standards, and so on, which never had much
> credibility to begin with.
Questionable conclusions weren't questioned enough - no reason to
repeat that mistake.
> We're all asking "when does method X work", and can't
> seem to agree on either the nature of any X or the meaning of "work".
> The "quantitative" studies mostly seem to have so many unknow
> variables that they add to the mess, rather than offer hope of
> sorting it out.
> What's to be done about all that ?
We appeal to reason! :-)
We avoid extreme skepticism; while preserving a healthy skepticism - we
reserve judgement until we have sufficient information to judge
responsibly. We avoid both cynicism and naive optimism. We avoid false
assumptions and strawmen and other forms of false argument.
http://www.randomhouse.com/catalog/display.pperl?1-4000-6171-7
Mr Martin may rightly say that strict separate phases are 'waterfall'
and Mr Roth may rightly say that strict waterfall may take place within
an iteration - we can resolve disputes about terminology.
Sometimes 'studies' are a secondary analysis of data gathered for some
ancient purpose - they just don't contain the data we need. Sometimes
'studies' are general questionaire responses - they just didn't ask the
specific questions we are interested in. We should look for 'studies'
that do contain the data we need; we should look for studies that ask
the specific questions we are interested in.
Isaac Gouy
> >> variables, with every study. If everything else fails,
> >> attribute the difference to personal factors. Using that
> >> tactic, you can nullify *any* conclusion.
> >
> > Ilja, have you looked at the study?
>
> No, I haven't.
I asked because it felt like you were saying that I was searching for
any possible reason to "nullify" the way the 'study' had been
interpreted.
Here's an analogy - the 'study' smells ;-)
> And just to be clear: I don't advice to fully trust any study.
>
> I actually don't think that anyone uses studies as there primary
> source for building opinions. At least I've never seen someone
> changing his opinion based on one.
Are your acquaintances really that closed-minded?
Ilja Preuß
>>>> They cannot be waterproof. There are always unknown
>>>> variables, with every study. If everything else fails,
>>>> attribute the difference to personal factors. Using that
>>>> tactic, you can nullify *any* conclusion.
>>>
>>> Ilja, have you looked at the study?
>>
>> No, I haven't.
>
> I asked because it felt like you were saying that I was searching for
> any possible reason to "nullify" the way the 'study' had been
> interpreted.
Sorry, that's not what I meant. Let me try to clarify:
I think you are right to be skeptical of this study. I think you would be
right to be skeptical of *every* study. A study is just that, a study, and I
don't think it can be done in a way to unquestionably proof anything.(*)
Still, I don't think it is reasonable to totally dismiss a study just
because it has flaws. (That is how your writing comes across to me.) I think
it's still an interesting data point - a data point that might (or might
not) connect to other data points, such as other studies, anecdotal reports
etc. to form a more complete, though probably still flawed picture.
>> And just to be clear: I don't advice to fully trust any study.
>>
>> I actually don't think that anyone uses studies as there primary
>> source for building opinions. At least I've never seen someone
>> changing his opinion based on one.
> Are your acquaintances really that closed-minded?
Not at all! Actually we seem to be quite successful in introducing new
ideas, trying new practices etc.
But it's not primarily the results of "objective studies" that let any of
them decide wether to do something or not. Much more important is wether an
idea
- matches personal experience,
- matches the personal value system,
- is recommended to us from someone we trust,
- etc.
Cheers, Ilja
(*) One problem is that the more you are trying to get the variables under
control, the more artificial, and therefore less significant to reality, it
becomes.
Isaac Gouy
> would be right to be skeptical of *every* study. A study is just
> that, a study, and I don't think it can be done in a way to
> unquestionably proof anything.(*)
I'm not trying to apply a standard of "unquestionable proof" - just
trying to build confidence that the arguments are sound.
Code is not just code - sometimes the code smells so bad that isn't
acceptable.
A study is not just a study - a study may connect evidence to
conclusions through sound reasoning; a study may be based on sound
reasoning but false evidence; a study may be based on sound evidence
but use faulty reasoning to reach a false conclusion, ... Sometimes the
reasoning smells so bad that isn't acceptable.
> Still, I don't think it is reasonable to totally dismiss a study just
> because it has flaws. (That is how your writing comes across to me.)
It's unreasonable to dismiss all studies because they may have flaws -
that's too skeptical and maybe cynical. It's reasonable to dismiss a
study because the conclusions do not follow from the evidence, or
because there's no link between the conclusions and evidence, ...
http://groups-beta.google.com/group/comp.software.extreme-programming/msg/9c3926c0069f59b9
> I think it's still an interesting data point
-snip-
How do you know if it's an interesting data point - you haven't looked
at it.
> - matches personal experience,
> - matches the personal value system,
> - is recommended to us from someone we trust,
> - etc.
Those are the 'reasons' that folk used to justify 'waterfall', burn
witches, ...
Ron Jeffries
>
>You're right - I am hopeful. Hopeful that the other studies have more
>substance; and hopeful that most people can look at evidence, notice
>the unknown variables, and realize how much that destroys the
>credibility of any conclusions we may wish to make.
Isaac, I'm suddenly seeing a common thread to many of your postings
over the recent past. I may have it all wrong. I'd like to describe
what I'm seeing here, because I'd like to understand what you're
doing, what you're looking for.
Much of what you say seems to have to do with destroying credibility.
It's as if you want the support for the things people are trying to
support not to be support after all.
I honestly can't tell if that's what's going on -- a desire to have
there be no support -- or if what you're looking for is some kind of
powerful support that you'll finally find satisfying.
And I honestly can't tell whether you have some personal strongly-held
beliefs about how software should be done, or not.
Can you enlighten me? I'm really quite interested.
Thanks,
--
Ron Jeffries
www.XProgramming.com
I'm giving the best advice I have. You get to decide if it's true for you.
Ron Jeffries
<newsg...@jhrothjr.com> wrote:
>> My definition of waterfall does not include the concept of increments.
>> Once there are increments, you aren't doing waterfall anymore, you are
>> doing incremental development.
>
>It's what you do inside the increments that's interesting.
>It's quite possible to do a monolithic waterfall inside
>a 3 month increment. You collect requirements for 18
>days, you design for 18 days, you code for 8 days,
>you test for so many days, then you integrate and so
>forth.
Yes, you're quite right, one could do that. Or inside the increment
you could collect requirements for 2 days, design for 2, code for 1,
etc.
There are two more or less independent notions here, that of
incremental/iterative, and that of what kind of phases, if any, are
used inside the iteration.
It's an interesting distinction that might help us with further
discussion.
>
>Incremental development gives you a short enough
>time frame for a reasonable likelihood of success.
>
>The project I mentioned in another post got pulled
>out of the fire when the last manager insisted on
>doing incremental development: deliver every three
>months. We still did requirements, then design, then
>split up to code and test, and then did integration
>at the end. It worked. That's not to say it couldn't
>have worked better: the PM in charge of the UI
>was always complaing about coordination with the
>batch process not happening right.
Yes. I know of one as well, and the RTF notion on my site is aimed at
the idea that with a relentless focus on delivery of Running Tested
Features, a team will become what we'd call "Agile" just because
there's no other way to sustain that focus.
>
>The confusion seems to be equating "Waterfall"
>with "excessively long lead time to delivery," or
>"one delivery per project." Maybe that's the way
>other people use it, but I don't see it that way.
The recent discussion in this thread has show (yet again) that using
the term "Waterfall" is fraught. Folks do not agree closely enough on
what it means, so that sometimes we get in trouble.
Yet when we get more specific, we may be talking about two different
ways of doing things, both of which can work just fine. It might even
be that the complete secret of success in software is to ship it in no
more than 3-month increments. I'd like to think that there's more to
it ... but that one step seems to do a lot!
Regards,
Ilja Preuß
> It's unreasonable to dismiss all studies because they may have flaws -
> that's too skeptical and maybe cynical. It's reasonable to dismiss a
> study because the conclusions do not follow from the evidence, or
> because there's no link between the conclusions and evidence, ...
So are you saying the latter is true for the report in question? From some
of your earlier posts I rather got the impression that the article was
missing important information to really assess the study?
As an aside, if I remember correctly, some time ago you were citing studies
from "Facts and Fallacies of Software Engineering" (thanks for the book tip,
BTW) - were you as critical with those? (No allegation implied, I'm just
curious.)
>> I think it's still an interesting data point
> -snip-
> How do you know if it's an interesting data point - you haven't looked
> at it.
I was talking about studies in general here.
>> - matches personal experience,
>> - matches the personal value system,
>> - is recommended to us from someone we trust,
>> - etc.
>
> Those are the 'reasons' that folk used to justify 'waterfall', burn
> witches, ...
That's exactly my point! They didn't do that because some studies showed
them that it was the right thing to do, and they didn't stop doing it
because some new study was published! It's simply not how humans typically
work - wether we liked them to, or not!
Cheers, Ilja
John Roth
"Ron Jeffries" <ronje...@acm.org> wrote in message
news:4te5v05nqv0a5ff8a...@4ax.com...
> On Fri, 21 Jan 2005 13:26:21 -0600, "John Roth"
> <newsg...@jhrothjr.com> wrote:
>
>>> My definition of waterfall does not include the concept of increments.
>>> Once there are increments, you aren't doing waterfall anymore, you are
>>> doing incremental development.
>>
>>It's what you do inside the increments that's interesting.
>>It's quite possible to do a monolithic waterfall inside
>>a 3 month increment. You collect requirements for 18
>>days, you design for 18 days, you code for 8 days,
>>you test for so many days, then you integrate and so
>>forth.
>
> Yes, you're quite right, one could do that. Or inside the increment
> you could collect requirements for 2 days, design for 2, code for 1,
> etc.
>
> There are two more or less independent notions here, that of
> incremental/iterative, and that of what kind of phases, if any, are
> used inside the iteration.
And that's the basic issue with Scrum: they don't define
what you do inside the sprints, so you can do anything
from strict phasist waterfall through XP - and I believe
I've heard tell of Scrum teams doing both.
Well, there seem to be two properties that are required.
Alistair Cockburn lists frequent deliveries as the first of the
seven properties required to get a project into the "safety zone".
For reference, they are:
1. Frequent Deliveries
2. Reflective Improvement
3. Osmotic communication
4. Personal Safety
5. Focus
6. Easy Access to Expert Users
7. Technical environment with automated tests,
Configuration Management and Frequent Integration.
He only requires the first three for Crystal Clear, but
strongly suggests the other four.
What's actually required is number 2 - as long as the
team discusses what worked and what didn't, and
is empowered to make whatever changes they want
on the basis of that discussion, the team will probably
get to a workable solution for them.
John Roth
Isaac Gouy
flaws -
> > that's too skeptical and maybe cynical. It's reasonable to dismiss
a
> > study because the conclusions do not follow from the evidence, or
> > because there's no link between the conclusions and evidence, ...
>
> So are you saying the latter is true for the report in question? From
some
> of your earlier posts I rather got the impression that the article
was
> missing important information to really assess the study?
When those information stepping stones aren't there,
leaping-to-conclusions may land you in fallacy.
> As an aside, if I remember correctly, some time ago you were citing
studies
> from "Facts and Fallacies of Software Engineering" (thanks for the
book tip,
> BTW) - were you as critical with those? (No allegation implied, I'm
just
> curious.)
I don't recall - presumably whatever I wrote is still there in the
newsgroup archive.
-snip-
> >> - matches personal experience,
> >> - matches the personal value system,
> >> - is recommended to us from someone we trust,
> >> - etc.
> >
> > Those are the 'reasons' that folk used to justify 'waterfall', burn
> > witches, ...
>
> That's exactly my point! They didn't do that because some studies
showed
> them that it was the right thing to do, and they didn't stop doing it
> because some new study was published! It's simply not how humans
typically
> work - wether we liked them to, or not!
Happily we don't need to take on responsibility for how others fail to
think - just for how we choose to think.
(That also saves us from pessimistic generalizations about the nature
of 'others'.)
Isaac Gouy
I'm motivated by curiosity. When someone draws my attention to
compelling studies and a book chapter titled iirc Evidence, I'll take a
look.
Why test programs? To verify that what we believe to be true
corresponds to some external reality.
Why test evidence and conclusions? ...
Gilligan
the term "Waterfall" is fraught. Folks do not agree closely enough on
what it means, so that sometimes we get in trouble.<<
That is a correct observation and I find it extremely interesting. The
waterfall approaches are supposed to be well defined. If they are well
defined then why don't we all understand them in the same way?
(Rhetorical)
So, are we tempted to define XP so that these confusions will not
continue? Someone once asked me about my Maverick method something like
this, "How can you define a method when you preach a model?" I
responded, "I hated to make a method, but I needed one as an
illustration."
So, let us define XP. Then if anything ocurrs that is not part of the
definition we can clearly say it is not XP! (NOT!!!) That would destroy
what I feel is the best part about XP (and similar methods). The
courage to improve or at least do your best!
So, do you define XP and make it non agile? Some will be tempted to do
so.
Geoff
Gilligan
Isaac Gouy
> The recent discussion in this thread has show (yet again) that using
> the term "Waterfall" is fraught. Folks do not agree closely enough on
> what it means, so that sometimes we get in trouble.
>
> Yet when we get more specific, we may be talking about two different
> ways of doing things, both of which can work just fine. It might even
> be that the complete secret of success in software is to ship it in
no
> more than 3-month increments. I'd like to think that there's more to
> it ... but that one step seems to do a lot!
John Roth
"Gilligan" <geoffrey...@gmail.com> wrote in message
news:1106497233.6...@z14g2000cwz.googlegroups.com...
XP has a reasonably clear definition. See either of Kent's books.
We have a lot of difficulty with people who say they're doing it,
are not doing it, and claim that their failures are the fault of XP.
> So, do you define XP and make it non agile? Some will be tempted to do
> so.
Some people will be tempted to do lots of silly things.
John Roth
>
> Geoff
>
Robert C. Martin
>> If you're doing XP, getting the requirements isn't a big issue.
>The Customer can get the requirements wrong.
Only for a few iterations.
Robert C. Martin
>> Here is the context of your quote from the study:
>Are you suggesting I quoted out of context?
>John Roth has made me hypersensitive ;-)
I don't think you were acting in bad faith; but I do think that the
broader context is worth considering.
>
>> Managers were also asked at which stage failure could occur...
>-snip-
>> The concept of a "requirements definition stage" is a waterfall
>> concept. Agile Methods do not have such a stage. So it seems
>> unlikely that any of the respondents who selected "requirements
>> definition stage" as a cause of failure would have been working
>> in XP, or an Agile Method.
>
>Let's say you're managing a project using XP, you feel that getting the
>requirements is a big issue (that is one of the reasons you're using
>XP), so which box do you tick when "asked at which stage failure could
>occur"?
In an XP project requirements are gathered and refined throughout the
process. The way those requirements are implemented is reviewed
continuously by business analysts, QA, and users. So the options
presented in those checkoff boxes don't apply well.
Isaac Gouy
> >The Customer can get the requirements wrong.
>
> Only for a few iterations.
The Customer sets a specific performance requirement, the system is
implemented at great difficulty and delay to meet that requirement -
when will anyone realize that the requirement is 10x more than needed?
(Meeting the performance requirement is one of the acceptance tests.)
Isaac Gouy
> >-snip-
> >> The concept of a "requirements definition stage" is a waterfall
> >> concept. Agile Methods do not have such a stage. So it seems
> >> unlikely that any of the respondents who selected "requirements
> >> definition stage" as a cause of failure would have been working
> >> in XP, or an Agile Method.
> >
> >Let's say you're managing a project using XP, you feel that getting
the
> >requirements is a big issue (that is one of the reasons you're using
> >XP), so which box do you tick when "asked at which stage failure
could
> >occur"?
>
> In an XP project requirements are gathered and refined throughout the
> process. The way those requirements are implemented is reviewed
> continuously by business analysts, QA, and users. So the options
> presented in those checkoff boxes don't apply well.
The only information that's reported is the managers response to those
"checkoff boxes that don't apply well".
That's the problem: we don't know from the report what "gap between
requirements definition and delivery" existed for these projects - 2
weeks or 2 years, we just don't know.
Otis Bricker
@c13g2000cwb.googlegroups.com:
I could be wrong since I don't practice XP but I thought XP advocated for
only implementing the items of greatest business value each iteration.
How did this performance requirement make it to the top of the list if it
isn't needed?
I can imagine that there might be a high impotance requirement that some
operation takes under some specific length. But when it causes the costs of
all the other requirements to double, wouldn't someone think its value
wasn't high enough to justify the total expense? Or compromise on a
slightly longer but less costly time? Or create two stories, one for less
than time 5x and another for less than x?
I know I would if I was guiding such a project.
--
Otis
John Roth
"Isaac Gouy" <ig...@yahoo.com> wrote in message
news:1106547493....@c13g2000cwb.googlegroups.com...
The critical point here is: great difficulty and delay. I think the
development team has the responsibility to point out that a
simpler architecture would cost less and be ready sooner, and
it also has the responsibility of painting a clear path from the simpler
architecture to the more expensive one, just in case it is actually
needed.
If the projected performance requirement is really ten times
what was needed when the application was deployed, then
one of two things has happened. Either marketing had its
head in the clouds, or the delay in implementation meant that
the application missed the window of opportunity, and someone
else got there first.
Or the application did not meet the real needs of the users,
so they don't use it very much.
John Roth
>
Isaac Gouy
http://groups-beta.google.com/group/comp.software.extreme-programming/msg/6bf81b1e6342d95a
> The critical point here is: great difficulty and delay. I think the
> development team has the responsibility to point out that a
> simpler architecture would cost less and be ready sooner, and
> it also has the responsibility of painting a clear path from the
> simpler architecture to the more expensive one, just in case it is
> actually needed.
As far as the Customer was concerned, "cost less and be ready sooner"
was about as interesting as getting a Cessna two seater when you asked
for a supersonic passenger jet.
As far as the Customer was concerned, the performance was required just
to support current trade volumes.
Work continued on other aspects of the system, more and more effort was
directed towards the performance issue, and the Customer became more
and more focused on the failure to meet a basic performance acceptance
test.
> If the projected performance requirement is really ten times
> what was needed when the application was deployed, then
> one of two things has happened. Either marketing had its
> head in the clouds, or the delay in implementation meant that
> the application missed the window of opportunity, and someone
> else got there first.
The Customer had made a simple mistake - they didn't remember something
correctly (the sort of simple factual mistake we have examples of in
this discussion).
> Or the application did not meet the real needs of the users,
> so they don't use it very much.
The application never went into production - it didn't meet the
acceptance tests ;-)
(And yes there were lawsuits, and yes I've changed details of the
story).
Isaac Gouy
for
> only implementing the items of greatest business value each
iteration.
>
> How did this performance requirement make it to the top of the list
if it
> isn't needed?
The Customer thought it was needed.
> I can imagine that there might be a high impotance requirement that
some
> operation takes under some specific length. But when it causes the
costs of
> all the other requirements to double, wouldn't someone think its
value
> wasn't high enough to justify the total expense? Or compromise on a
> slightly longer but less costly time? Or create two stories, one for
less
> than time 5x and another for less than x?
Performance X really was needed just to support existing business - but
the Customer didn't remember X correctly, and thereafter always refered
to their own notes rather than to the original source of information.
Isaac Gouy
"All of us, as far as I can remember, thought waterfalling of a huge
project was rather stupid, or at least ignorant of the realities."
Gerry Weinberg
-- Page 81. Agile and Iterative Development, a Managers Guide.
Craig Larman.
And
'The article quotes Jerry on the Mercury Project as saying "...
waterfalling a huge project was rather stupid ...". That jarred as I
remembered Jerry saying things somewhat differently. I later checked
the threads and QSM4. Sure enough, on page 316 Jerry says "Always use
the Waterfall Model /when it applies/." The italics are his. Then the
adjective kicked in: "... a *huge* project ..."'
http://www.ayeconference.com/wiki/scribble.cgi?read=WaterfallDejaVu
I haven't checked what it says on page 316 of QSM4
http://www.dorsethouse.com/books/qsm4.html
Gilligan
methodologies is not to do them.
Robert C. Martin
>The Customer had made a simple mistake - they didn't remember something
>correctly (the sort of simple factual mistake we have examples of in
>this discussion).
Teams correct mistakes like that (just as the team here corrected
factual errors on the newsgroup). If the customer made a factual
mistake on the order of 10X at a high cost, *somebody* would notice.
If nobody noticed, then the whole team is so dysfunctional that
nothing worthwhile will ever get done.
Isaac Gouy
> methodologies is not to do them.
The strange thing is that there must also be few? many? who think that
they should do 'waterfall' - otherwise what's all the fuss about?
Robert C. Martin
In an XP environment there would be a story that says: "Meet
performance goal X". If it involved "great difficulty and delay" then
the estimate on that story would be large.
When customers pick stories to be developed in the next iteration,
they consider both the business value and the cost estimate. If the
cost estimate is high, it may not fit inside a single iteration. When
this happens the story must be split into many stories. It may be
that this story would be split into:
"Meet performance goal X/10".
"Meet performance goal X/5".
"Meet performance goal X/2".
"Meet performance goal X".
Each of which is estimated to take less time than one iteration, but
each of which must follow the other in succession. (i.e. they depend
on each other in order)
If there are stories that have higher business value for lower cost,
they will be chosen first. It seems unlikely that the customer would
drive all the way up the ladder from X/10 to X without noticing that
such high performance wasn't necessary.
In the end, however, there are some customers who utterly mismanage
the requirements. If I had to tick a box to describe why such a
project failed, I'd tick the management box.
Otis Bricker
@z14g2000cwz.googlegroups.com:
>> I could be wrong since I don't practice XP but I thought XP advocated
> for
>> only implementing the items of greatest business value each
> iteration.
>>
>> How did this performance requirement make it to the top of the list
> if it
>> isn't needed?
>
> The Customer thought it was needed.
And the most important item to implement right now given the cost the
team gave him?
>
>> I can imagine that there might be a high impotance requirement that
> some
>> operation takes under some specific length. But when it causes the
> costs of
>> all the other requirements to double, wouldn't someone think its
> value
>> wasn't high enough to justify the total expense? Or compromise on a
>> slightly longer but less costly time? Or create two stories, one for
> less
>> than time 5x and another for less than x?
>
> Performance X really was needed just to support existing business - but
> the Customer didn't remember X correctly, and thereafter always refered
> to their own notes rather than to the original source of information.
>
>
So rather than check, the Customer was willing to put up with all of that
developement overhead the team told him about? And no one estimating that
story asked for clarification?
How would this be done better in a different process? Why assume the
Customer would know any better there? Honestly, it is easier for me to
imagine this happening with BUF requirements than XP style ones. If it
makes it into the up front requirements, who will question it later?
It may be of limited imagination but could you provide a specific example
of the type of problem you propose?
--
Otis
Ron Jeffries
wrote:
>Thanks. I think many feel that the thing we learned from Waterfall
>methodologies is not to do them.
That's a close approximation to what I learned. There are /techniques/
in waterfall that I'd use. But if by waterfall we mean a strict single
pass through analysis, design, code, and then test ... I'd never do
that again.
Isaac Gouy
> >methodologies is not to do them.
>
> That's a close approximation to what I learned. There are
/techniques/
> in waterfall that I'd use.
Such as...?
Isaac Gouy
> So rather than check, the Customer was willing to put up with all of
that
> developement overhead the team told him about? And no one estimating
that
> story asked for clarification?
The Customer kept giving them the wrong answer :-)
> How would this be done better in a different process? Why assume the
> Customer would know any better there?
Good questions!
In reverse order:
- the Customer wouldn't know any better, and would continue to make the
same simple mistake
- (the process need not be that different) requirements need to be
justified in a systematic way, with reference to primary source
material (so they can be cross-checked).
Isaac Gouy
something
> >correctly (the sort of simple factual mistake we have examples of in
> >this discussion).
>
> Teams correct mistakes like that (just as the team here corrected
> factual errors on the newsgroup).
Sorry, it doesn't feel like there's been much 'teamwork' correcting
factual errors on the newsgroup.
What did Laurent write: 'I fear that "most people" are not as diligent
as you are in going back to primary sources.'
> If the customer made a factual mistake on the order of 10X at a
> high cost, *somebody* would notice.
> If nobody noticed, then the whole team is so dysfunctional that
> nothing worthwhile will ever get done.
Actually a lot got done - but the spurious requirement caused the
project to be cancelled; client spent a few millions USD, consulting
company went out of business, lawsuits followed.
John Roth
> This is a fun digression, the main point is here
> http://groups-beta.google.com/group/comp.software.extreme-programming/msg/6bf81b1e6342d95a
>
>
>> The critical point here is: great difficulty and delay. I think the
>> development team has the responsibility to point out that a
>> simpler architecture would cost less and be ready sooner, and
>> it also has the responsibility of painting a clear path from the
>> simpler architecture to the more expensive one, just in case it is
>> actually needed.
>
> As far as the Customer was concerned, "cost less and be ready sooner"
> was about as interesting as getting a Cessna two seater when you asked
> for a supersonic passenger jet.
>
> As far as the Customer was concerned, the performance was required just
> to support current trade volumes.
>
> Work continued on other aspects of the system, more and more effort was
> directed towards the performance issue, and the Customer became more
> and more focused on the failure to meet a basic performance acceptance
> test.
This is one of two issues. I'm seeing an architectural problem
here. While it isn't always true, my general rule of thumb is
that when people are struggling with performance issues, and
they've done the obvious things, there's an underlying architectural
issue: the architecture on the selected hardware won't handle the
load.
Without knowing the system involved, it's pointless continuing
the analysis.
>> If the projected performance requirement is really ten times
>> what was needed when the application was deployed, then
>> one of two things has happened. Either marketing had its
>> head in the clouds, or the delay in implementation meant that
>> the application missed the window of opportunity, and someone
>> else got there first.
>
> The Customer had made a simple mistake - they didn't remember something
> correctly (the sort of simple factual mistake we have examples of in
> this discussion).
>
>> Or the application did not meet the real needs of the users,
>> so they don't use it very much.
>
> The application never went into production - it didn't meet the
> acceptance tests ;-)
Which is the other problem. Without a real world test, we will
never know whether the customer was right or not. Everything
else is an assumption.
>
> (And yes there were lawsuits, and yes I've changed details of the
> story).
Which is why it's pointless continuing the examination of the
example.
John Roth
>
Ron Jeffries
Was this an agile project?
Ron Jeffries
Well, for example ... Asking questions about requirements. Drawing
pictures of objects to figure out what the design is. Running the
debugger. Figuring out how to test things. Writing documents.
Reviewing code. All kind of things. I just don't do any one of them
for a long time, and I strive never to do any of them unless and until
I really need to.
Phlip
> Well, for example ... Asking questions about requirements. Drawing
> pictures of objects to figure out what the design is. Running the
> debugger. Figuring out how to test things. Writing documents.
> Reviewing code. All kind of things. I just don't do any one of them
> for a long time, and I strive never to do any of them unless and until
> I really need to.
That's not recommendable. I like diagrams.
The goal is this: Diagrams, debuggers, and documents no longer control us.
We take control of when and how we indulge in them. We have more options.
--
Phlip
http://industrialxp.org/community/bin/view/Main/TestFirstUserInterfaces