Bugless development (was Eliminate waste)
Alexander Aizikovsky
> If you eliminate bug tracking you hide bugs, which must be fixed.
> Hiding bugs does not do your system bugless.
The suggestion was to eliminate bugs, not the bug tracking system. Bug
tracking should have died by itself if there had been no bugs.
I was on a project where total number of filed bugs was over 4000.
More than 500 of them were in Open state, moving from release to
release without any chance to be fixed.
If bug exists in a system for a certain period, we must be very
careful to count it still as a bug.
Not on one occasion fixing an old bug had considered by customers as
introduction of new and unwanted behavior, and has been reverted back
to original state.
Recently our development tried working much closely with testers.
When user story comes to a reviewable state, a pair of developer and
tester would spend 30-40 min playing with the story.
It usually was enough for a tester to find a couple of dozens of
issues. Some of them so simple that they have been fixed during the
session. Others were considered as highest priority for the team.
We continued logging more complicated bugs into the bug tracking
system, because our QA manager did not feel comfortable of not doing
so. For years number of found bugs was indirect measure of QA
efficiency (rather than development inefficiency)
Well, we did not achieve zero bug level, but it was a very promising
step forward.
Does anybody have similar experience?
BTW.Successful Software book by James Shore has a chapter called No
bugs: http://www.jamesshore.com/Agile-Book/no_bugs.html
I would recommend the chapter and the whole book to everybody who
explore ways to succeed in software development.
Regards,
Alex
ArtyRak
> More than 500 of them were in Open state, moving from release to
> release without any chance to be fixed.
Some questions to clarify the situation:
1. Did you distinguished "bug" and "defect"?
2. Did you deliver features with that bugs (4000)? Did customer use
them now?
3. Did you manage a huge, complicated project without any chance to
break down it on smaller projects (products) AND (it means in the same
time) iterations?
4. How did you confirm that product design was right (because it looks
like design mistakes)?
5. How many features was delivered to customer without open or delayed
bugs?
6. And finally, how did you estimate such a huge project? How customer
was convinced in it's future success?
After answers on this questions, we will be able to continue this
discussion :)
On 26 мар, 03:31, "Alexander Aizikovsky" <aiz...@gmail.com> wrote:
> On Mar 25, 11:28 am, "Vasyl Keretsman" <vasi...@gmail.com> wrote:> If you eliminate bug tracking you hide bugs, which must be fixed.
> > Hiding bugs does not do your system bugless.
>
> The suggestion was to eliminate bugs, not the bug tracking system. Bug
> tracking should have died by itself if there had been no bugs.
>
> If bug exists in a system for a certain period, we must be very
Alexander Aizikovsky
> > I was on a project where total number of filed bugs was over 4000.
> > More than 500 of them were in Open state, moving from release to
> > release without any chance to be fixed.
>
> IMHO, "something is wrong in the kingdom" :)
Finding what's right and more important how to make it better is a
more interesting problem.
> Some questions to clarify the situation:
> 1. Did you distinguished "bug" and "defect"?
No. Everything was a record in bug tracking system. I don't understand
the difference.
The word defect sounds better :)
> 2. Did you deliver features with that bugs (4000)? Did customer use
> them now?
Of course.
BTW it is a very successful product. The company became a leader in
North America on a very competitive market and is expanding its
business to Europe, Asia and Australia.
> 3. Did you manage a huge, complicated project without any chance to
> break down it on smaller projects (products) AND (it means in the same
> time) iterations?
It was a chain of relatively small (~2-3 months) projects all related
to the same product.
> 4. How did you confirm that product design was right (because it looks
> like design mistakes)?
The design, as well as vision of the product and understanding of the
domain has been changing continuously. And still is as far as I know.
You are right of course. There were problems with design. We called
those points 'bug farms'. In each release we were targeting not only
new features, but also known design issues.
Once we have had a release completely dedicated to redesign, but I
don't think it is a good practice.
> 5. How many features was delivered to customer without open or delayed
> bugs?
I doubt there were any.
During 4 years of development once we had to roll back the upgrade.
And we issued ~5 patches. I don't have exact numbers at hand. One of
the patches was rather an unplanned release - new feature was needed
urgently. Others were bug fixes. The overall number of bugs reported
by customers in production did not exceed 20. Not sure in this number.
Maybe it was 30.
> 6. And finally, how did you estimate such a huge project? How customer
> was convinced in it's future success?
The way you eat an elephant - piece by piece :)
The first release was in production in 4 month after development
started. And then pretty much every 2-3 months.
Each time delivering new market-winning features.
One release was about 8 month long. Attempting to deliver more than
usual, we ended up delivering very little in this release.
It is in house development. The development team is a part of the
company.
>
> After answers on this questions, we will be able to continue this
> discussion :)
My pleasure.
Don't know if my answers are relevant. It would be easier for me if I
knew the point you trying to prove or deny, so I could focus on
specifics.
Thanks
Alex
borys....@gmail.com
I'm working in project where total number of bugs were estimated
to 10000. The main purpose of the work is to support and develop
system with very large redundancy. And I think that 4K bugs is not
deadly bad. But 500 OPENED bugs is really dangerous. This means that
bugs should be rearraged ASAP, because it means lost of navigation in
project ...
A propos, in our team only thing that differ bug, defect and
change proposal is priority. All those things considered as tasks for
developer, irrespective to the type
ArtyRak написав:
ArtyRak
> domain has been changing continuously. And still is as far as I know.
> You are right of course. There were problems with design. We called
> those points 'bug farms'. In each release we were targeting not only
> new features, but also known design issues.
> Once we have had a release completely dedicated to redesign, but I
> don't think it is a good practice.
So! This IS the answer :)
And the description of "defect" - the behavior of function (feature)
that leads to some result, but not to the perfect one, or not in the
perfect way. Function (feature) works with some proviso - this is not
bug, but misunderstanding OR change of requirements after release of
feature.
As I understand, you made chain of projects (and one product) without
any Change Control. You fixed Change Requests as bugs - this is
widespread mistake :)
So, You wanted to drive project without any bug to track down? You
should get it with appropriate Change Control (it starts after any and
each product release) :)
I don't want to dive deeper in your case. Maybe I'm right, maybe not.
But I write down my guess, and You can develop it to appropriate
conclusions. :)
Wish You good luck in the war against bugs :)
Alexander Aizikovsky
On Apr 1, 11:26 am, "ArtyRak" <Arty...@gmail.com> wrote:
> > The design, as well as vision of the product and understanding of the
> > domain has been changing continuously. And still is as far as I know.
> > You are right of course. There were problems with design. We called
> > those points 'bug farms'. In each release we were targeting not only
> > new features, but also known design issues.
> > Once we have had a release completely dedicated to redesign, but I
> > don't think it is a good practice.
>
> So! This IS the answer :)
Sorry, you lost me. Answer to what question?
> And the description of "defect" - the behavior of function (feature)
> that leads to some result, but not to the perfect one, or not in the
> perfect way. Function (feature) works with some proviso - this is not
> bug, but misunderstanding OR change of requirements after release of
> feature.
This does not sound clear to me. I've been using Google toolbar for a
long time. Its current version provides instant suggestions as you
type. It is 'more perfect' than typing without suggestions, but I
would not say that the previous behavior had a 'defect'.
I don't think though that the difference between bugs and defect
matters.
Whatever product owner wants to do now, should be done now.
PO has to make his mind on what exactly needs to be done next: a bug
fixed, a defect removed, a feature implemented.
Just a single thing.
> As I understand, you made chain of projects (and one product) without
> any Change Control. You fixed Change Requests as bugs - this is
> widespread mistake :)
In fact we do have a separate log for change requests. And planing
board on their biweekly meetings prioritizes the requests. They are
rarely implemented. Seems like waste to me.
>
> So, You wanted to drive project without any bug to track down? You
> should get it with appropriate Change Control (it starts after any and
> each product release) :)
I don't see a big difference between bug tracking and change control.
As mentioned above, product owner should decide on one thing to do,
and instead of backlog, change request or bug tracking system this
single item should go straight to development.
This approach is called "one piece flow" versus "batch and queue",
which applies to backlogs, bug tracking systems and Change request
logs.
The terminology comes from Lean development. See for example:
http://www.leanadvisors.com/Lean/glossary/
> I don't want to dive deeper in your case. Maybe I'm right, maybe not.
'My case' actually was given as an example of how ineffective huge
queues are. And how expensive there maintenance could be. 'One piece
flow' seems like a better approach to me.
> Wish You good luck in the war against bugs :)
Thanks :)
ArtyRak
>
> As mentioned above, product owner should decide on one thing to do,
> and instead of backlog, change request or bug tracking system this
> single item should go straight to development.
The difference is straight enough. After revision of change requests
(before development starts, if there is any CR to develop), you must
revise ALL requirements to check inconsistency and you must redesign
solution (and revise existing features) to provide coverage for
already implemented requirements. After this activities, PO have to
approve CR and corresponding changes in features (in product backlog).
This is the main difference between bug tracking and Change Control
(and also Requirements Control). You can fix bug "in one piece", but
you can not always fix CR "in one piece".
And this is the reason for me to say "So! This IS the answer :)". You
postponed redesign, and thus postponed "bugs" which was in fact CR.
But in the PO's mind, product WAS "redesigned", because he already
"fixed" (at least recognized and approved) the "bug".
"- Just a small bug - nothing extraordinary. We can fix it later in
the next "one peace flow"..." - he said.
Maybe, I describe my opinion in some foggy way... But for me is
natural to track changes and bugs separately. And when I hear "- To
fix it, we need redesign" - this sounds as the bell and means "- We
actually have some requirement changes". And this is the time to stop
and revise everything closely. Because the product design itself is
based on requirements.
Alexey Krivitsky
it seems to me you're going in loop
and in fact there is no right approach (surprised? :))
there are just least and most effective ones that are actually can
become quite opposite
when taken to another context...
one thing i'd like to stress is that being agile (doing lean) means to
be effective in terms of
being fast at delivering quality products that are requested by the market.
whatever this means in your particular cases
there are though some proven rules that Alexander Aizikovsky stressed.
such as reducing of queues may increase performance throughput.
i can only confirm that this worked in my cases.
apart from them it is up to you if you have 2 queues: one fore
features, one for bugs,
or a single one. what matters is to be effective.
----
also a remark on a term "Control of requirements".
guys, you cannot control them, who tried already failed.
why? because there are no requirements. full stop.
this was provoking, but indeed if requirements can change over time,
then in fact it had not been requirements at the first place. it was
rather someone's ideas
and predictions, which one cannot control. just can help evolve in the
adequate direction.
ArtyRak
> guys, you cannot control them, who tried already failed.
> why? because there are no requirements. full stop.
>
> this was provoking, but indeed if requirements can change over time,
> then in fact it had not been requirements at the first place. it was
> rather someone's ideas
> and predictions, which one cannot control. just can help evolve in the
> adequate direction.
I can hardly agree with this :) You may handle requirements or do not,
but they exist. And first of all, in the brain of Product Owner.
Obviously, everything is changing on Earth, requirements too. If you
do not know about this changes, it does not mean that they are
unnecessary. And also they WILL have an influence on project work flow
(if you want it or not). The form of requirements can be different but
they also differ from "ideas". Requirements have to be implemented and
ideas - not. The difference is the decision of PO - to implement
something. Thus without any dependence from the form of requirement,
the decision of it's implementation ought to be controlled. Because
decisions lead to responsibility. And, Yes, the formality level of
this "requirements control" ought to be selected by responsible person
- PO. And it can be "the help to evolve in the adequate direction" :)
And when this responsibility for decisions lead to "500 open bugs", I
think that formality level was not selected properly.
Alexander Aizikovsky
On Mar 31, 1:47 pm, borys.leb...@gmail.com wrote:
> Hi!
>
> I'm working in project where total number of bugs were estimated
> to 10000. The main purpose of the work is to support and develop
> system with very large redundancy. And I think that 4K bugs is not
> deadly bad. But 500 OPENED bugs is really dangerous.
Did you fix all the 10,000 bugs?
Do you think that all the bugs must be fixed?
Well, our bugs were not 'opened', they were officially 'suspended'.
Not sure if it helps.
Either way, people wasted time on recording, reading and other ways
maintaining error/defect log.
Alexander Aizikovsky
On Apr 5, 8:47 am, "ArtyRak" <Arty...@gmail.com> wrote:
> I can hardly agree with this :) You may handle requirements or do not,
> but they exist. And first of all, in the brain of Product Owner.
> Obviously, everything is changing on Earth, requirements too. If you
> do not know about this changes, it does not mean that they are
> unnecessary. And also they WILL have an influence on project work flow
> (if you want it or not). The form of requirements can be different but
> they also differ from "ideas". Requirements have to be implemented and
> ideas - not. The difference is the decision of PO - to implement
> something. Thus without any dependence from the form of requirement,
> the decision of it's implementation ought to be controlled.
I agree with everything you said up to this point.
>..Because
> decisions lead to responsibility. And, Yes, the formality level of
> this "requirements control" ought to be selected by responsible person
> - PO.
PO needs visibility and transparency.
But being an business domain expert she probably does not know details
of the development. The team including PO has to decide on the best
way to provide enough visibility and control. There are different
approaches:
Formal requirement management
Burndown chart
One item at a time.
I was advocating the last one. Unfortunately I don't have positive
experience, just some negative :(, but it still looks very attractive
to me. If working software is upgraded with new features, defect
fixes... every week, there is no need for PO to maintain any
requirements backlog.
>And it can be "the help to evolve in the adequate direction" :)
> And when this responsibility for decisions lead to "500 open bugs", I
> think that formality level was not selected properly.
You are right, 500 bugs is an indication of a problem. Maybe formality
level is the one.
During our retrospectives, this specific reason did never come up. But
there were several dozens of other suggestions. We tried some of them.
For instance, the biggest impact on bug number per release was made by
pairing of a tester and a developer early in the project.
I try to advertise a process when people do not spend time on filling
up bug tracking systems.
Instead they focus on delivering new features, fixing problems,
exploring the system's behavior and doing other stuff that helps to
develop better software and simply is more fun than writing bug
descriptions.
The next paragraph is slightly off topic. What is the goal of software
development?
500 bug/defects is not necessarily a bad thing. As I said, the project
was and still is a success. There are examples of extremely successful
software with known bugs/defects. Go to http://support.microsoft.com/search/
and select your favorite product.
Cheers,
Alex
ArtyRak
> up bug tracking systems.
> Instead they focus on delivering new features, fixing problems,
> exploring the system's behavior and doing other stuff that helps to
> develop better software and simply is more fun than writing bug
> descriptions.
And with this phrase "do not spend time" we turn back to "bugs" and
CR :)
You DO NOT have to spend much time on tracking CR - you spend time on
REDESIGN, which sounds better. :)
And you spend ONLY one "peace of time" for one CR - larger then for
"one bug", but smaller then "one bug every time" (every Scrum) :)
Think about law of conservation of energy. You will spend a time on
idea/bug/CR/"crutches"/organizational decision in any case, but you
can choose the most effective way.
And, yes, you can make successful product with bugs. But... This is
the question of Quality Control - you can agree with "low quality" (I
mean 500 bugs) when it satisfies user/customer/PO :) And, at last,
this IS the quality in fact - satisfaction of the customer. You can
just close this 500 bugs. Why not? The only thing you have to do - is
to define quality requirements for product BEFORE starting development
(iteration?). This will close such a "bugs" (let name them cases)
automatically.
> software with known bugs/defects. Go tohttp://support.microsoft.com/search/
Alexander Aizikovsky
On Apr 5, 1:57 pm, "ArtyRak" <Arty...@gmail.com> wrote:
> The only thing you have to do - is
> to define quality requirements for product BEFORE starting development
> (iteration?).
This sounds interesting.
If you decide to answer the next question, let's start a new thread.
How do you define quality requirements?
Definitely it's not the number of bugs. 500 bugs != low quality. I'm
sure that if you had a chance to use the product you would agree.
Customers' perspective was actually that we spent much more time on
increasing the quality than they (customers) needed.
Business people were begging to lower the quality, but deliver
features faster.
We tried many different ways of defining "just enough quality", none
of them seems working.
What do you do?
Thanks,
Alex
ArtyRak
First of all, I'm sorry for my "pure netiquette" and my pure
English :)
So, going back to Quality. What does Quality mean? There are a lot of
answers, but I'd prefer the formal one - http://en.wikipedia.org/wiki/Quality
"Degree to which a set of inherent characteristic fulfills
requirements" as ISO 9000.
So, If your customer agree that application can crash once a day -
this is the quality requirement ("try not to crash more than once a
day in usual conditions" - the stronger one). This is very simple
example. For large product we can allow sticky functions (and even
function groups) and, in the other hand, we will gain the profit by
providing cutting-edge product. And this is NOT a low quality product.
This IS the quality, to provide such a product just in time.
Thus, when we agree that 500 "bugs" are allowed in market version of
product, we can just CLOSE that bugs (non-usage-critical sort of
quality requirements). Finish.
But to accept this decision, we ASAP HAVE to accept quality
requirements to measure out this "allowed" bugs :)
No requirements == no decisions.
No Requirements Control == no prioritizing instrument for decisions.
No Change Control == no signal to decide, if we prioritized "bugs" in
the right way (no way to quantify our mistakes).
No difference between bug and CR == no way to organize redesign
activities effectively == avalanche effect from one single "bug"
without timely redesign.
There are obvious rules for me as PM, developer and PO (in some
sense).
The practical side of quality requirements.
Just quantify the time of user spent on daily routines with some set
of features. If this user spend the same time (minus learning) after
bugs fixed for this features - there is no obvious reason to fix the
same bugs next time. So, you can compare bug with "useless" one and
automatically postpone if similar, and after some cool-down period
with no second report on this bug - close it. This is empiric sort of
quality requirements.
The stress tests give us the third sort - non-mission-critical quality
requirements. If we can steer stress test to the end - we obviously
have NO mission-critical bugs. Other existing bugs can be "fixed"
without any development - e.g. with organizational decisions.
Do not mix "the quality" and "the sort". The quality (of product
features) defines the sort (of bugs), but not vice versa :)
Sincerely Yours,
Artur Rakytskyy