"ideal time" in scrum?

[dotnetguy]

What is the purpose of "ideal time" in Scrum?  Isn't "ideal time" an
alternative to story points?  Are story points also theoretically
supposed to be estimated for "ideal conditions"?  If so then what's
the benefit of using that technique as opposed to realistic estimates?

[RonJeffries]

Andrew,

Ideal time is not a Scrum concept. Story points are not a Scrum concept. Stories themselves are not a Scrum concept.

All of these things are used by some people as part of their work with Scrum.

Ideal time is an alternative to story points. There are at least these ways to estimate stories:

• Estimate how long the story will take to accomplish in elapsed time;
• Estimate in "ideal time", the time you think it would take if everything went well and you were not interrupted;
• Estimate in "story points", which are relative numbers in comparison to other stories.

They are all ways of estimating how long the story will take to do. Generally speaking, you can't use any of them directly:

• Elapsed time estimates tend not to average at zero error. Generally they are optimistic, but not always.
• Ideal time estimates have the above problem and you also have to take how many real hours it takes to get an ideal hours' work done.
• Story points estimates also tend not to average to zero error. And you have to have a factual or estimated idea about how long it takes to get a story point done.

Again, none of these are part of Scrum. All of them are used in some Scrum projects in some places.

Ron Jeffries

www.XProgramming.com

Sometimes I give myself admirable advice, but I am incapable of taking it.

-- Mary Wortley Montagu

[dotnetguy]

"With ideal time estimates you have to take how many real hours it
takes to get an ideal hour's work done"

I'm not clear on how this description defines the difference between
an ideal hour and a real/actual hour?

Is an ideal hour an hour with no dependencies or delays?  If a task
involves dealing with 1 or more third parties then I'm not sure how
the concept of ideal hours would handle this?  ex - you need to have
meetings with the third parties, you need to integrate with the third
parties, test the integration and fix any issues.  When you're dealing
with new third parties there's no way to really know how long it's
going to take until you start having discussions.  The third party may
have different ideas about how the implementation should be done, the
third party may have additional requirements, the third party may have
skill or resource gaps that the primary party would not know about up
front.

An "ideal" hour for third party integration would be telling the third
party that for example you need a web service that will return a
certain response based on a certain request.  You would spend 15
minutes to put this request in an email and you would get a response
from the third party  providing instructions on how to get exactly the
information you want from a brand new custom web service that they
build just for you.

However, 30 minutes "ideal time" for third party integration is not
realistic so I don't see how this type of estimate adds any value.  In
fact, this type of estimate seems to add risk by creating unrealistic
schedule expectations.

Please let me know if I don't understand the concept of ideal time
correctly or if what I'm describing is an example of a flaw with ideal
time estimates....

On Sep 30, 7:25 pm, RonJeffries <ronjeffr...@acm.org> wrote:
> Andrew,
>

[Yves Hanoulle]

Scrambled by my Yphone

Op 1-okt.-2012 om 06:12 heeft dotnetguy
<andrew.d....@gmail.com> het volgende geschreven:

> "With ideal time estimates you have to take how many real hours it
> takes to get an ideal hour's work done"
>
> I'm not clear on how this description defines the difference between
> an ideal hour and a real/actual hour?
>
> Is an ideal hour an hour with no dependencies or delays?

Time you are not disturbed to work on the story

> If a task
> involves dealing with 1 or more third parties then I'm not sure how
> the concept of ideal hours would handle this? ex - you need to have
> meetings with the third parties, you need to integrate with the third
> parties, test the integration and fix any issues. When you're dealing
> with new third parties there's no way to really know how long it's
> going to take until you start having discussions. The third party may
> have different ideas about how the implementation should be done, the
> third party may have additional requirements, the third party may have
> skill or resource gaps that the primary party would not know about up
> front.
>
> An "ideal" hour for third party integration would be telling the third
> party that for example you need a web service that will return a
> certain response based on a certain request. You would spend 15
> minutes to put this request in an email and you would get a response
> from the third party providing instructions on how to get exactly the
> information you want from a brand new custom web service that they
> build just for you.
>
> However, 30 minutes "ideal time" for third party integration is not
> realistic

although that when you add your testing time, that is exactly the
touch time you had.

> so I don't see how this type of estimate adds any value

The value lies in the fact that this makes it possible to more compare
the stories.
the nr of interruptions tends to be more
Depending on the company/team then the stories.

> . In
> fact, this type of estimate seems to add risk by creating unrealistic
> schedule expectations.
>

They all have that risk.
A big difference is the discussions around difference between ideal
time and ellapsted time can be done separate from the story
discussion.

Yes some (most) companies have an unrealistic idea of how many ideal
hours they can do in one day.

In some companies I had seen, it's less then 4 a day.
All managers will acknowledge that you can't have 8 ideal hours in a day.
Unfortunately most will want you to come close.
With ideal time, you can separate that discussion from the development
time discussion.

> Please let me know if I don't understand the concept of ideal time
> correctly or if what I'm describing is an example of a flaw with ideal
> time estimates....
>
>
> On Sep 30, 7:25 pm, RonJeffries <ronjeffr...@acm.org> wrote:
>> Andrew,
>>
>> On Sep 30, 2012, at 9:23 PM, dotnetguy <andrew.d.ciccare...@gmail.com> wrote:
>>
>>> What is the purpose of "ideal time" in Scrum? Isn't "ideal time" an
>>> alternative to story points? Are story points also theoretically
>>> supposed to be estimated for "ideal conditions"? If so then what's
>>> the benefit of using that technique as opposed to realistic estimates?
>>
>> Ideal time is not a Scrum concept. Story points are not a Scrum concept. Stories themselves are not a Scrum concept.
>>
>> All of these things are used by some people as part of their work with Scrum.
>>
>> Ideal time is an alternative to story points. There are at least these ways to estimate stories:
>> Estimate how long the story will take to accomplish in elapsed time;
>> Estimate in "ideal time", the time you think it would take if everything went well and you were not interrupted;
>> Estimate in "story points", which are relative numbers in comparison to other stories.
>>
>> They are all ways of estimating how long the story will take to do. Generally speaking, you can't use any of them directly:
>> Elapsed time estimates tend not to average at zero error. Generally they are optimistic, but not always.
>> Ideal time estimates have the above problem and you also have to take how many real hours it takes to get an ideal hours' work done.
>> Story points estimates also tend not to average to zero error. And you have to have a factual or estimated idea about how long it takes to get a story point done.
>>
>> Again, none of these are part of Scrum. All of them are used in some Scrum projects in some places.
>>
>> Ron Jeffrieswww.XProgramming.com
>> Sometimes I give myself admirable advice, but I am incapable of taking it.
>> -- Mary Wortley Montagu
>

> --
> You received this message because you are subscribed to the Google Groups "Scrum Alliance - transforming the world of work." group.
> To post to this group, send email to scruma...@googlegroups.com.
> To unsubscribe from this group, send email to scrumallianc...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/scrumalliance?hl=en.
>

[RonJeffries]

Andrew,

On Oct 1, 2012, at 12:12 AM, dotnetguy <andrew.d....@gmail.com> wrote:

"With ideal time estimates you have to take how many real hours it
takes to get an ideal hour's work done"

I'm not clear on how this description defines the difference between
an ideal hour and a real/actual hour?

Is an ideal hour an hour with no dependencies or delays?  

The notion of an ideal day was invented to help people get in touch with their gut feel about doing some work: "I could do that in a day if everyone would get out of my way." Its purpose was to focus on how long the thing takes to DO, factoring out all the meetings, interruptions, and so on.

Note in passing that I am DESCRIBING ideal days, which we invented on the C3 team, for XP, fifteen years ago. I am not recommending them. I really don't recommend estimation at the level of individual stories and Sprints at all. It's not necessary, it doesn't help much, and it results in hours and hours spent thinking about estimation.

I understand that you want to think about and do estimation ...

If a task
involves dealing with 1 or more third parties then I'm not sure how
the concept of ideal hours would handle this?  ex - you need to have
meetings with the third parties, you need to integrate with the third
parties, test the integration and fix any issues.  When you're dealing
with new third parties there's no way to really know how long it's
going to take until you start having discussions.  The third party may
have different ideas about how the implementation should be done, the
third party may have additional requirements, the third party may have
skill or resource gaps that the primary party would not know about up
front.

Ideal days were invented to measure doing WORK. You are describing, primarily, DELAYS. 

Delays occur in actual time and do not move the project forward. Ideal time is good enough when the effect of delays is roughly constant, so that the ratio between ideal days and real days is roughly constant. The situation you describe does not sound as if the delays have a constant effect. I'd expect the time we have to wait to get a meeting would vary widely; the time they would take to think about things and get back to us would vary widely, and so on.

So if you have a value stream full of delays, and you want to know how long things will take, ideal time will not help you much.

In truth, knowing exactly how long it would take to code and test the thing would not tell you much either. The problem isn't your estimates of the work, it is the existence of delays and the way we work with them.

An "ideal" hour for third party integration would be telling the third
party that for example you need a web service that will return a
certain response based on a certain request.  You would spend 15
minutes to put this request in an email and you would get a response
from the third party  providing instructions on how to get exactly the
information you want from a brand new custom web service that they
build just for you.

Ideal hours do not apply here. They apply to estimating how long to do the software development. This is not software development. This is administration. CURB will not help us with this. Ideal time will not help us with this. Developers who magically know how long it will take to code anything will not help us with this.

However, 30 minutes "ideal time" for third party integration is not
realistic so I don't see how this type of estimate adds any value.  In
fact, this type of estimate seems to add risk by creating unrealistic
schedule expectations.

Correct. No form of estimation is ideal (sic) for this issue. Elapsed time estimates may help somewhat, but we will find that they are inaccurate as well.

A value stream analysis and/or a task board will better help us deal with this kind of situation. Stories or tasks will pile up in various "waiting for review" columns. (A really brave team would set WIP limits so that when too many things are waiting for review, all work would stop.)

The issue before us is not estimation, it is work flow. Our work is interrupted unpredictably, by events that take a long time compared to how long it takes to actually do the work. 

Scrum will tell us that without the need for estimates. At the daily standup:

Yesterday I coded X and send it to Third Party;

Today I can code Y and send it to Third Party;

In my way is that I am waiting for A, B, C, D, ..., to come back from Third Party. I think I have forgotten how A works ...

Please let me know if I don't understand the concept of ideal time
correctly or if what I'm describing is an example of a flaw with ideal
time estimates....

It is not a flaw in a Philips screwdriver that you cannot repair a tooth with it. Ideal time estimates are a tool for getting in touch with how long a developer needs to implement something. They are fairly good for that. Nothing is really good for estimating how long it will require to implement something, and no amount of ritual will make the estimates better.

What's worse is that improving estimates almost never improves the project. Generally, it makes the project worse, because the easiest way to be sure we always meet our estimates is to make them larger, and then do the work to match. The result of this is that everything takes longer. In addition, if the organization is focused on "meeting" estimates, when the developer starts coming up on the estimated time, he or she begins to try to cram all the remaining work into the remaining few minutes. There goes testing, there goes keeping the code clean. Here come defects, and here come slowdowns tomorrow as we work with this dirty code.

When I encounter a team obsessing over estimates, my experience is that usually I am also encountering a team with several other likely large problems:

• Often they do not work well together, because it is more important for an individual to meet his own estimates than to help someone else;
• Often they have a long and growing defect list, because rushing to hit a deadline produces bugs;
• Often they find that their code "needs refactoring", because rushing produces inferior code.

I said usually. Truth is, I have never met a time-obsessed team where the obsession wasn't a sign of a deeper problem. There might be teams out there who are producing clean, bug-free code like clockwork, but somehow just can't  figure out how long it will take to do anything. I've just never seen one ...

Ron Jeffries

www.XProgramming.com

Don't ignore your dreams; don't work too much; say what you think; cultivate friendships; be happy. -- Paul Graham

[Dan Rawsthorne]

realistic estimates of what? And what does realistic mean?

Dan Rawsthorne, PhD, PMP, CST
3Back.com
Author of Exploring Scrum: the Fundamentals

[dotnetguy]

@Paul - I thought during Sprint planning the goal was to estimate in
realistic hours not ideal hours.

Ideal hours assumes everything goes perfectly which is not realistic.


On Oct 1, 9:17 am, paul mahoney <mahoneypa...@gmail.com> wrote:
> Andrew,
>   On the teams that I have coached or been a scrum master for I use both
> story points and ideal time for different levels of estimating.  We use
> story points for estimating the size of stories. The story
> point estimates allow a team to determine what work can be accomplished in
> a release and a sprint.
> Please keep in mind this is an inexact science so to speak, story points
> are estimates and estimates very from team to team and project to project.
>
> During the sprint planning session we decompose the user stories into high
> level tasks and estimate the tasks in ideal hours.  this ensable the team
> to better estimate the suer stories they can complete in a sprint.
>
> As others have mentioned the process mentioned above is not strictly
> "Scrum" but I have seen these estimating processes have been
> extremely beneficial for the teams I have worked with.  As with all things
> in the Agile world one should inspect and adapt.
> Paul

[dotnetguy]

@Ron - I like your perspective on how third party integration should
be estimated as workflow.  Can you please elaborate on that?

[Kurt Häusler]

Estimating in real hours assumes estimating in real hours is possible,
which is not realistic. It also raises questions about why the
estimated real hours don't match actual real hours, which they should
if they are real.

Not that I am a fan of estimating "ideal hours" either.

Wouldn't it be better to use concrete measurements instead, when it
comes to time spent?

[RonJeffries]

Andrew,

On Oct 2, 2012, at 1:18 AM, dotnetguy <andrew.d....@gmail.com> wrote:

@Paul - I thought during Sprint planning the goal was to estimate in
realistic hours not ideal hours.

Ideal hours assumes everything goes perfectly which is not realistic.

First:

Scrum does not require estimation.

At all.

In Sprint Planning, the team considers the Product Backlog Items which the Product Owner proposes for the upcoming Sprint.

The team ensures that they understand the items well enough. They discuss how they will do them. They decide how many they can accomplish. They commit to take on that many items and to accomplish them, producing a Product Increment that meets the team's current Definition of Done.

That is all. Scrum does not require estimation at all. 

Some teams choose to estimate backlog items. 

This can be one decent way to decide how much work to take on.

Some teams choose to track estimated vs actual time.

This practice offers little value and can have serious negative consequences.

Some teams do it anyway. 

Some may even get value from it.

Scrum does not require it.

If a team chooses to estimate, 

which is not required,

one way to do that is to use ideal time.

Ideal time is easier for some teams to use for estimation,

because you just think about how long it would take to do the thing without interruptions

but taking into account everything else you know about the problem and the code.

To decide how much work to take on in the Sprint:

add up the ideal time accomplished in preceding Sprints;

take on work summing to approximately that amount of ideal time again,

allowing for any special circumstances that are predictable.

For example:

OK, this one is a six, these four are all threes, these three are twos. Twenty-four ideal hours.

We've done between twenty-two and twenty-eight ideal hours the last few Sprints.

This upcoming Sprint, Dave is on vacation, and we have that all-hands meeting. 

That's going to knock out about 1/8 of our progress because of Dave, and another 5 percent for the meeting;

Call it twenty percent. So we'll probably do between seventeen and twenty-two.

Twenty-two is iffy. Let's commit to the six and the three fours, for eighteen. 

We'll take on some of the twos if things go well.

Let's do it.

Note that none of this is deep, complex, nor requires much accuracy.

In Sprint Planning, we just decide how much work to take on.

The only point to estimating is to decide how much work to take on.

To decide how much work to take on,

assess the amount of work by any consistent means;

compare to recent performance;

adjust for anything obvious;

commit to that much.

That is all.

During the Sprint, we want to know how we're doing, and whether there is any item at risk. 

If items are at risk, 

we may need to take corrective action, 

or work with the Product Owner to set expectations or adjust the plan.

At no time during this process do we need accurate estimates, 

and there is little value to getting them.

At no time during this process do we need to convert estimates to actual, 

and there is little value to doing so.

Why is there little value to accurate estimates?

Because most of the things that make reality not match expectation 

are not known, and 

cannot be known, 

at the time of planning.

Scrum is about setting a reasonable, short term plan which is probably attainable,

committing to accomplish it under reasonable circumstances,

and managing our actions in accord with what actually happens.

In Scrum we do not call our shot and make it. 

We set our direction, guess about how far we'll get, and steer to get there.

We do not need accurate estimation to do that.

We scarcely need estimation at all.

We steer as we go.

Scrum is about steering, not about predicting.

Ron Jeffries

www.XProgramming.com

[RonJeffries]

Andrew,

On Oct 2, 2012, at 1:20 AM, dotnetguy <andrew.d....@gmail.com> wrote:

@Ron - I like your perspective on how third party integration should
be estimated as workflow.  Can you please elaborate on that?

I do not favor estimation.

Let me put it another way: I do not recommend estimation. 

The concern with third party integration is that much of it, the waiting, and the meetings, and the further waiting, is all impediments to getting done. They are not productive work. No, really. They are not productive work. They are waste.

What do we do with waste?

We remove it.

We draw a picture of our workflow. It might look like this:

Wait for spec;

Get spec;

Wait to read it;

Read it;

Wait for meeting to figure it out;

Meet to figure it out;

Wait for some testers to help;

Meet with testers to think up tests for it;

Wait for Sprint Planning meeting;

At Sprint Planning, plan it into a Sprint;

Type in the code; 

Wait until someone has time to type in the tests;

Type in the tests;

Wait for testers to run the tests;

Testers run the tests;

Wait to write up results and put in Jira;

Write up results, put in Jira;

Wait until someone reads Jira;

Someone reads Jira;

Wait till time to call tester and ask WTF;

Call tester to ask WTF;

Get voicemail;

Wait for tester to call back;

Finally meet with tester, find out WTF;

Loop back multiple times until code seems to work;

Wait to submit code to third party;

Submit;

Wait till third party batches up enough stuff to be worth meeting;

Meet with third party;

DISCOVER THAT WE DIDN'T UNDERSTAND THE SPEC

Loop;

OK, maybe it isn't quite that bad. Or, maybe it isn't quite that good. Much of the stuff above is not work, it is waiting. Much of the waiting then causes rework, as we loop back to fix our tests found by testers or to fix whatever we didn't understand.

Either way, there is no point estimating this mess. The thing is to fix the mess. For example:

Express and share specifications in form of tests. 

When the tests run, the code is done.

This eliminates much of the waiting for the 3p, and shortens the meetings when they are required.

Write our own tests up front as part of Backlog Refinement.

Programmer codes until tests work.

This eliminates waiting for testers to type things in, waiting for Jira, waiting to talk to tester, and eliminates most looping for rework.

And so on. Many of the things you want to estimate are actually impediments. Impediments should be removed, not estimated.

[Pierre Neis]

Hi,

What do you want to measure with real time estimates?

If you don't want to reinvent the wheel in a make-this-sense matter then limit your WIP.

I agree that this kind of estimates are powered by magic: predict the future. But, if I did understand my practice, Scrum is about to evaluate effectiveness.

Enhancing your scrumboard with Kanban techniques (sorry for the purists) and try to figure out team capabilities, improve it and limit the work-in-progress (WIP) gives you a better overview (in my POV).

In May, I did send warnings to CapGemini during a Scrum Program Audit. They gives ideal times estimates and that gave these results:

- after 1 week: overtime is standard

- after 1 month: +10% turnover in development teams, program in danger (flagged red at Management Committee)

- after 6 weeks: Management (my customer) forced offshore going nearshore (single location with business and management)

- after 8 weeks: no overtime, no ideal time only Sprint Backlog estimates

- after 10 weeks: program on track

- after 2 months: program delivered one week late

Conclusion:

- provider did use ideal time to sale productivity

- productivity metrics was all wrong because of bad focus: high productivity, low features delivered, increasing program risk by opacity

- Actuals / Estimates are enough and should be reviewed at MetaScrum/Sprint Review/Portfolio Meeting to clear Definition-of-ready

- Estimates are high level based on Release Plan and should be improved at Sprint Review in large transparency

- providers always delivers optimistic estimates!

Pierre E. Neis Scrum/Lean Improver WE & Co , the Collab Lab |  PLöRK  Mobile: (+352) 661 SCRUMS Luxembourg - Brussels - Paris - Zürich - London - Amsterdam - Barcelona - Hanoi   http://meetwith.me/pierreneis

 

Owner of the "Product Owner's Help Desk" - "PLöRK - the leader's tool"

Contact me: pierre.neis

Get a signature like this. CLICK HERE.  

[dotnetguy]

@Ron - There may be stories that require third party integration.  A
Scrum team will usually treat this story as any other story as far as
sizing and estimation.

A workflow perspective could help with sizing/estimation.  I know you
don't like estimation but if all stories are sized/estimated then
stories involving third party integration will also need to be sized/
estimated.

So the question is - in an environment where all stories need to be
sized/estimated what's the best way to size/estimate stories involving
third party integration?  It seems like a workflow perspective could
be helpful.  The question is what's the best way to integrate this
workflow perspective into the Scrum sizing/estimation process for
stories involving third-party integration?

On Oct 2, 1:46 am, RonJeffries <ronjeffr...@acm.org> wrote:
> Andrew,
>

[RonJeffries]

On Oct 2, 2012, at 12:45 PM, dotnetguy <andrew.d....@gmail.com> wrote:

@Ron - There may be stories that require third party integration.  A
Scrum team will usually treat this story as any other story as far as
sizing and estimation.

Integration yes. Coordination with the actual 3P people, which you seemed to me to be talking about, not so much. Taking on a story where we don't know what we have to do is a bad idea. Therefore those meetings and approval steps are waste. Therefore they are impediments and should be address.

A workflow perspective could help with sizing/estimation.  I know you
don't like estimation but if all stories are sized/estimated then
stories involving third party integration will also need to be sized/
estimated.

If all noses must be cut off, then we must cut off our noses. Still the wrong thing to do ...

So the question is - in an environment where all stories need to be
sized/estimated what's the best way to size/estimate stories involving
third party integration?  It seems like a workflow perspective could
be helpful.  The question is what's the best way to integrate this
workflow perspective into the Scrum sizing/estimation process for
stories involving third-party integration?

I don't know and honestly don't care what the best way is to estimate a bad process. 

I'd work to fix the problem, not to estimate my way out of it.

I suppose one estimates by observing the past and using that to predict. 

One might schedule fixed times for meetings, fixed durations for them, and so on. 

This would give an appearance of knowing what was gong on. 

It would still be an inferior approach compared to removing the need by reordering the workflow.