#NoEstimates puzzle experiment in Melbourne

Craig Brown and I ran a variant of Chris Chapman’s now famous jigsaw puzzle experiment at a meetup of the Melbourne Agile and Scrum User Group on Wednesday evening.

Everyone had fun and was intensely engaged throughout. There were loads of interesting dynamics emerging from the teams, perhaps surprising given the contrived nature of the experiment.

Set up

  • We set up three same-sized (10-12 people) teams, each with:
    • an identical jigsaw puzzle (way too big to be completed)
    • a Product Owner (to provide the vision and direction) and
    • a Scrum Master (to help the team achieve the PO’s vision)
  • We opted for 3 * 15-minute iterations, with 3 minutes for a Retro in between
  • Each team was told to use a different method – one was a Scrum team, one was a “mob team” and one was a “no rules” team. Here’s what that meant:

Scrum team

  • Must have Planning (including estimation), Review and Retro in each iteration
    • We provided Planning Poker cards for the estimation but the team was free to choose whatever estimation method they liked
  • Must only work on “stories” agreed in Planning – new stories can’t be introduced mid-iteration
  • Stories are only “done” when PO accepts them (in Review or before)

“Mob” team

  • No formal ceremonies required
  • Team all works on one story at a time until “done” (single-piece flow approach)
  • No estimation
  • Retro encouraged but not “enforced”

“No Rules” team

  • Can work like the Scrum team, the Mob team, any combination of the two, or any other way they like

Outcome

  • 600_301537052600_301537062Scrum team delivered most stories (3; the other teams delivered 2 each)
  • Whole group was asked to vote on which they thought was the best outcome
    • “No rules” team won (emphatically)
    • Scrum team lost

Interesting Observations

Here are some empirical observations of the evening’s events and outcomes, along with my interpretation of what they indicate in an Agile/#NoEstimates context (==> in bold italics underneath the observation).

Scrum team

  • Delivered most in terms of stories but least in terms of value, both for their Product Owner and as voted for by the wider group
    ==> Output <> Value
    ==> Comparing teams in a useful way would require consistent measures of both effort and value velocity across teams
  • Spent far too large a proportion of time (particularly the first iteration) in planning, and needed to be alerted to this fact
    ==> Consistent timeboxing is important to ensure there is time to do all that is required, and for less variability of outcomes
  • A member of the team openly admitted that he inflated an estimate because he did not agree with the value of the story that the PO wanted to do next
    ==> Estimates are often gamed, and for various reasons

“No rules” team

  • Implicitly chose not to estimate, but instead to maximise the time they had for building
  • Eventually delighted their Product Owner (and wider group), but during the game the PO felt like:
    • The approach to delivery was too ad-hoc, even chaotic, especially at the beginning
      ==> Teams must collaborate in order to be co-ordinated, improve and deliver the right outcomes
    • Stories were too large (epic) so delivery all happened near the end rather than incrementally
      ==> Smaller stories have lower variability and can help with early and frequent delivery, creating better predictability for PO/customer and lessening the need for estimates
      ==> Larger, higher variability stories rely on estimates of time, or at least relative size, to provide the illusion of predictability
  • Started with no process at all but this was deemed unproductive (with such a big team), so they split into smaller teams with focused goals
    ==> Smaller teams are more effective because it is easier to collaborate, change direction, gain consensus, etc.

General

  • Scrum and Mob team both delivered purely incrementally (concentrating on edges) rather than iteratively (identifying a recognisable area of interest and building upon it), although stories were clearly too big
    ==> An iterative approach is crucial for risk management, predictability and delivering the right thing (value), i.e. without such an approach you have no choice but to estimate
  • Product Owners all felt like they weren’t being listened to – this had particularly bad consequences for the Scrum and Mob teams, perhaps due to their purely incremental approach
    ==> Important for all team voices to be heard, especially given the PO is driving what should be built in order to deliver on the vision

Scrum Basics Part 2 – Monitoring Progress Toward a Goal

"Various projective practices upon trending have been used to forecast progress, like burndowns, burn-ups, or cumulative flows. These have proven useful. However, these do not replace the importance of empiricism. In complex environments, what will happen is unknown. Only what has happened may be used for forward-looking decision-making."

-- Scrum Guide

Agile/Scrum teams are often asked to estimate how long a release might take. Or an entire project. Sometimes this is done under the guise of relative size estimates like T-shirt sizes – or, perhaps more commonly, story points – coupled with an estimated (or guessed) velocity. This is sometimes done even with new teams that have no velocity history.

Scrum, as defined in the Scrum Guide, places a large emphasis on the use of empiricism. Aside from the quote above, the following nuggets can also be found:

"Scrum is founded on empirical process control theory, or empiricism. Empiricism asserts that knowledge comes from experience and making decisions based on what is known. Scrum employs an iterative, incremental approach to optimize predictability and control risk."
"[The Scrum Master helps the Product Owner with] Understanding product planning in an empirical environment"

My interpretation of Scrum is that, while the Development Team are expected to estimate each PBI (Product Backlog Item), they are not asked nor expected to determine delivery dates, or how much work will be completed by a delivery date.

At Sprint Review:

"The Product Owner discusses the Product Backlog as it stands. He or she projects likely completion dates based on progress to date (if needed)"

So, the Product Owner uses the estimates on the PBIs combined with the empirical knowledge gained from what has actually been done to determine completion dates of a set of PBIs (e.g. a release). At no point does the Product Owner ask the team what will get done (beyond the current Sprint).

This use of empiricism is often neglected by Scrum teams. Teams are asked to project release dates, sometimes several months out, without any velocity history. This is not making projections based on what has actually happened. It is not empirical, and does not work in a complex, ever changing environment.

"A Product Backlog is never complete. The earliest development of it only lays out the initially known and best-understood requirements. The Product Backlog evolves as the product and the environment in which it will be used evolves. The Product Backlog is dynamic; it constantly changes to identify what the product needs to be appropriate, competitive, and useful. As long as a product exists, its Product Backlog also exists."

If you are using estimates, it is important that you use probabilistic estimates based on real, empirical data. Scrum suggests this. Practitioners suggest this also. Don’t ask the team to forecast any further out than the current Sprint. As the Product Owner, use real data to make forecasts and decisions. Asking the team to make longer term projections is not respecting the data showing what is actually getting done.

Monitor progress, don’t try and predict it.

Stop using time in relative size estimates!

The SAFe approach to normalised story points makes a classic mistake that everyone seems to make with story points. It is not “relative sizing” to compare stories to a reference story that has been estimated in time (in this case “about a day”).

As soon as you introduce time as a basis for your reference story, and use e.g. story points with Fibonacci sequence, all of the comparisons you make are based on time, i.e. a 2 point story pertains to 2 days, 5 points to 5 days, etc.

Even if you are not doing this consciously you will do it unconsciously. So all you have done is estimated “how long” the stories will take to deliver. This is not relative sizing!

The whole point of using relative sizing instead of time-based estimation is that humans are better at comparing the size of things than we are about making absolute judgement of size, e.g. we’re good at being right that building A is bigger than building B, but we’re not so good at being right that building A is about 200 metres high and building B is 150 metres.

Unfortunately when it comes to tasks that we perform, our natural tendency is to use absolute terms because the “size” of a task essentially equates in our brains to “how long”. The fact that story points are numbers doesn’t help with this. Where story points completely lose their value is when we start deliberately equating a point value with a length of time.

True relative sizing of a backlog is to pick a low value story (one that you are unlikely to implement for some time) and do not estimate it at all. What you now do is compare other stories to that story, i.e. I think story C will take longer than story B, story D will take longer than story C, story D is about the same size as story C, etc.  At no point do we actually predict how long something will take. We are simply saying which stories will take longer than others, by our estimation.

When a new story emerges you then do the same thing – decide if it will take longer than the reference story (which, because you have not yet implemented it, you will not be influenced by the actual time it took), less time or about the same.

You can now measure progress against the total backlog as you deliver the stories.

One thing I do agree with in the SAFe approach is that you should not do any re-calibration/estimation. As soon as you start re-estimating stories based on how long things are actually taking you are being influenced by time. This can not only throw off the relative calibration of the backlog but also ignores the inherent variability of software increments; i.e. there will be outliers within size groups that take significantly longer (or shorter) than the modal average.

P.S. If you’ve read my other #NoEstimates stuff on this blog you will know I do not advocate the use of story point estimations at all, especially due to the way they are typically misused and abused. However, there may be some potential value in doing relative size estimates (e.g. T-shirt sizes), if done right, for one or more teams working from the same initial product backlog in order to give some indication of the overall viability of the initiative and to provoke discussion within the team(s) about the value and possible approaches for undertaking individual pieces of work, aka “what shall we do next”.

#NoEstimates – An Alternative Means of Risk Management

Introduction

A continuing theme of counter-arguments posed at the #NoEstimates ideas is that development cost estimates are required in order both to manage risk and to derive value.

This blog post intends to give further insights into how risk can be effectively managed, and how we might determine the value of our initiatives, without the need for making up front and deterministic development cost estimates.

Risk

“Risk is the probability of an unfavorable impact to the project” – Glen Alleman (@galleman).

From the risk angle, the argument goes along the lines that the built-in “risk management” in Agile approaches is not aligned with conventional definitions of risk management in software development.

I’ll go along with this. Agile (and #NoEstimates) does not take the conventional approach to software risk management, which sees project success as “on time, on budget” and thus requires an up front estimate of total scope, cost and duration.

Agile/#NoEstimates offers an alternative way to manage risk on projects (and, no, I’m not taking about Agile Estimation, the spin-off brand of traditional estimation promoted by Mike Cohn). I’ll explain more about this later.

Value

The argument regarding value is that estimated cost is required to determine value, given that value is related both to the timing of when things are released and how much it costs to develop the things that will (potentially) generate value. That the worth of something to someone can only be evaluated if we know how much that things costs.

Again I agree to an extent, but there are two key sticking points for me here. One is that we only know how much software development costs after the fact. People say “we need to estimate because we need to know the cost”. Estimating, however accurately we think it is being done, does not allow us to know the cost.

Before the event we can only estimate what will be done and how much it will cost. In addition, the further out we are estimating cost and value, the riskier (and potentially costlier) our estimates become.

By estimating, rather than fixing, cost we have no greater insight into the value, which is also estimated. Essentially we are increasing our risk by estimating both cost and value rather than just value, which is what #NoEstimates promotes. More on this later.

The other sticking point is that value is often highly subjective and personal. I know how valuable a particular brand new Ferrari is, partly because I know how much it costs. That said, if you gave me two different Ferraris to test drive and didn’t tell me how much they cost, I would tell you which one I prefer. Which one was more valuable to me. This has nothing to do with the cost. The one I prefer might be significantly cheaper, but its value to me is higher because it’s more fun to drive and I prefer the look of it.

The same applies with software. There is so much to consider when we try and measure value. Aside from the empirical measure of monetary returns, we have to consider the needs of the customers, the stakeholders and our corporate strategy (to name but a few), not to mention the fact that all of these things change over time.

Agile is about delivering value early, not trying to predict how to maximise value over a given timeframe or a product’s lifecycle. It is the early delivery of value that allows us to tune and adjust our course for maximum longer term benefit.

This is why it is an alternative, and completely viable, approach and should be considered as such.

Agile Risk Management

The key aspects of Agile that help us manage risk effectively are:

  • Iteration
  • Continuous selection of highest value work (i.e. making decisions)
  • Fixed, cross-functional teams with 100% focus on current project
  • Early and frequent delivery of end-to-end working software increments and
  • Empirical measures of progress toward goals

Waterfall-computer-wallpaperWith Waterfall projects, the need for conventional risk management is clear. We have no way of measuring progress from day one in terms of working software because we are carrying out requirements analysis, specification and design phases before we write a line of code. People are often working on multiple projects and so we must allocate a percentage of their time to the project at hand.

The only way to measure percentage progress toward project completion is to have a breakdown of the SDLC phases and tasks within each, estimated in days/weeks, and tick them off as we go along. If we don’t complete all the necessary tasks for a given phase in the estimated timeframes, we are “off track” and we need to take corrective action.

With a phased delivery approach, the only way to manage risk is to have an estimate of the total scope, cost and duration of the project.

But if we are working in an Agile way, we are not taking a phased approach to project delivery. We are delivering full end-to-end working solutions in an iterative manner, early and frequently. We are working in fixed, cross-functional teams so teams costs are known and consistent.

This approach allows us to manage risk and measure progress toward project completion (meeting of stakeholder goals within a given budget) from the get-go.

Progress

If we are truly iterating by delivering vertical slices through the system, after our first iteration we will be able to measure progress toward the project goals. We will have delivered a working, albeit perhaps low quality, solution to the problem. We may even have actually met the project goals.

Either way, we can inspect what we have done and decide if we are on the right track. If we are, we can iterate over our solution, improving quality in the desired areas and incrementing new features. If we are not, or we see a better way of solving the problem, we can throw away what we’ve done and start again. We may even decide to scale up our efforts and add more teams, if there is emergent estimated value in doing so.

Given in Agile we are delivering end-to-end working software from the get-go, we are not burdened with the problems we faced in our Waterfall projects for measuring progress. We have the ability to empirically measure progress because we are delivering “done” functionality, as opposed to hitting pre-determined “milestones” which are not based on what we have actually delivered in terms of a working product.

green-traffic-lightIn Waterfall, so long as we are hitting our milestones then the project status is “green”. For software product development projects, this means that we are deferring our risk management until we actually start writing code. We don’t know that the scope of what we want to build is achievable, and we can’t reduce scope until we actually realise it’s too much (well into the development phase, deep into the project).

In Agile we can manage scope right from the beginning, because we are continually focusing on building the most valuable thin, vertical slices which represent iterations over an end-to-end solution to the problem. We can empirically measure how much we got done and how much is left to do. We can regularly take proactive decisions to cut scope or switch to an alternative approach to improve our chances of delivering a successful outcome. What should we do next for maximum value and maximum impact in meeting our goals? What should we not do? What is the simplest approach for our next iteration?

This is risk management.

These kinds of conversations enable us to focus on doing the simplest thing, for maximum impact, given the budget that we have available. To not wait 9 months to deliver a solution but to deliver a solution in 1 month, then make it better.

mona_jonah-640x209Most “Agile” projects are not managing risk

If we decide up front in a project inception on the requirements (product backlog) and solution we will be sticking to, and estimate it will take, say, 9 months, all we will do is incrementally build the solution, usually in horizontal slices, components or modules.

After each “iteration” we will not have a holistic view of what we’re building.

This is a very common approach by “Agile” teams. In this situation we are deferring the management of risk until we actually have a system that can meet (some of) the needs of the project stakeholders, usually late in the game when the deadline is getting close.

This is not risk management. If we work in this way we cannot work with #NoEstimates.

How do we estimate value without estimating development cost?

OK, so assuming we have the capability and will to deliver vertical slices through a solution early and rapidly, and we have a fixed cross-functional team, 100% committed to the project at hand, we can focus on the potential value of the ideas we want to build while controlling cost using small “drips”.

When we use ROI to decide whether a project is worth pursuing, or which of 2 or more potentially valuable projects we should choose given limited people and resources, we base the “investment” measure on the estimated cost, of which the development costs are part, and the “return” is the value we expect to generate, measured on the same scale as the investment (usually money).

There is a flaw with this approach.

6 months, 2 years, it’s all the same!

Let’s say we estimate a project will take 6 months of development time, costing $500k. We expect that when the product is complete it will generate $2m in revenue. The timing of when that revenue gets generated is key. Will we get anything at all before the product is built in its entirety? Will there be a few months of marketing required after all the features are done before we will start seeing the cash rolling in?

The implication of the timing of value generation is that the actual ROI of what we’re building in a 6-month project might still be negative after 6 months of development time, even if we get everything done that we originally wanted done (and estimated).

Now compare that to, say, a project with an estimated duration of 2 years. After 6 months, the ROI of the two projects will be identical. Our net loss in both cases is $500k, so our ROI is -100%; we have spent half a million bucks with nothing (yet) to show for it.

So, given the erratic, inconsistent and numerous ways we can measure value in software, is the traditional ROI approach an ideal decision making model in this domain?

mortazavi20110206103527187Agile is about early delivery of value, not trying to predict maximum value

The upshot of this is that the less risky approach to generating a positive “ROI” is to work on options that will potentially generate value early, i.e. with relatively small and simple effort. Put simply, if we prioritise initiatives by virtue of which ones we expect to generate value early rather than how much value they will generate over the product’s lifecycle then we do not need to batch these initiatives up into “projects” and estimate how long the project will take.

This can easily be reverse engineered. If our starting point is a “project”, with a list of requirements, the best thing we can do to manage risk (keep our decisions within the bounds of the near, more certain, future) and ensure we deliver value early is to pick the most valuable requirement/problem to solve and come up with a simple, creative approach to fulfilling that requirement in a very short timeframe.

What’s next? One at a time…

The team can go away for, say, 1 month, after which time we holistically assess where we’re at in terms of fulfilling that requirement. What have we learned? Is this requirement still the most valuable one to work on (ignoring sunk costs)? Are we better off ditching what we’ve done and investing in attacking another requirement?

Our measure of what is valuable must reset after each iteration. It’s irrelevant how much we’ve already spent (sunk cost fallacy).

We need to constantly concern ourselves with what is the most valuable thing to do next. This is Agile. This is #NoEstimates.

And this is risk management. Yes, it’s an approach that requires a different way of thinking about how we choose what work to invest in, how much to invest and the decisions we make along the way. But it is risk management nonetheless.

RealEstateBlog360-BigMoneyBut we can’t do this when $200m is at stake!

The #NoEstimates debate has hit a point where the main remaining arguments are around its application in big money projects. Most of the original dissenters – who have now spent time reading more about the ideas put forward by myself and the other #NoEstimates crew – are now in agreement with us that, at least for small scale projects, we can get away with not doing “micro-estimates”, and indeed it may be preferable to work this way.

But when it comes to “macro-estimates” – i.e. how much of the customer’s money are we going to spend – it is argued that a #NoEstimates approach is not viable. That when “you are spending someone else’s money” you need a plan (estimated schedule) to ensure you deliver what is required for the money, with some deterministic level of confidence.

The irony of this argument is that when the big number guys come out swinging with their big numbers, these numbers are estimates! When we call a project that we haven’t yet completed, or even started, a “$200m project”, what we are actually saying is “our customer has a $200m budget and we have to deliver what they want for their money”. In other words, the decision has been made to go ahead, and the budget is $200m. There is no go/no-go decision to be made – it’s already been decided that the project is going ahead, and they want a result for $200m.

For me, with such large sums and timeframes at play, there is all the more reason to manage risk by drip funding small amounts and iterating over a solution in the way I’ve described. Scaling up where required. Tuning and adjusting.

The alternative is to manage risk by using probabilistic estimation techniques based on past projects such as Monte Carlo simulations to derive a total estimated cost with a confidence interval, and then constantly adjust these calculations as the project progresses. But I maintain that the Agile way, where we start from a budget or fixed deadline and then actively build and manage scope along the way, is preferable because it harnesses the creativity of designing and building great software and allows us to welcome and embrace change every step of the way.

Quote-33Create the future rather than predict it

Instead of trying to nail down a plan and predict outcomes, we are forging our own future based on current market conditions at any given time, and the way we feel about what we’ve built so far. We are controlling our costs by working with fixed teams in short timeboxes, and we are constantly assessing the value of what we’re building.

If we work this way we do not need to estimate things up front. Empirical data is being generated as we go along, and we can look at the market with fresh eyes after each iteration. We can see what we’re getting done and what we’re not. We can change our mind on whether we care that we didn’t get the things done that we wanted to get done. We can see which of our assumptions were true and which were false. We can steer our ship in whichever direction we need to avoid the iceberg ahead, while remaining focused on the destination.

This is at the heart of #NoEstimates from my point of view. It is possible to work this way. It is not easy to get to a position where you are able to, but if you can get to that place it is, as Ron Jeffries describes it, “the best known way to work”.

Chris Chapman “interviews” me about #NoEstimates :)

1. You’ve mentioned on Twitter that in your opinion, #NoEstimates = Agile + Real Options. For the curious newbie, what does this mean?

The approach I talk about is very much underpinned in Agile principles. In fact it’s what I believe Agile was intended to be at its core (although I’ve had some disagreement from the likes of Ron Jeffries and Alistair Cockburn on this point).

To summarise #NoEstimates from my point of view:

Constraints breed creativity

  • Use real constraints to drive decisions, e.g. “this is how much we want to spend” or “we need something by June in time for Wimbledon”
  • Arbitrary constraints (such as deadlines based on zero to low knowledge “estimates”) cause dysfunctional and ineffective behaviour
  • Create mini-constraints (i.e. drip funded iterations) to promote a creative approach to what we are going to build to address the problem at hand

Build awesome teams

  • Create fixed, capable teams so we know how much our time costs
  • Scale up team capacity if enough positive value has emerged (by adding teams, not people to teams)
  • Empower our teams to be bold and free in making solution choices, with focus on “building the right thing” and “delighting customers and stakeholders”

Keep our options open

  • Cover multiple, potentially valuable options with small experiments rather than committing to one option per team for long periods
  • Reassess options frequently to ensure initiative is still valuable (ignore sunken cost) and is more valuable than other options for which we could divert our team capacity
  • Anything we haven’t yet built (e.g. our product backlog) is only an option – we shouldn’t assume we’ll build it and shouldn’t worry how “big” it is unless we actually want to do it now, or very soon

Put the “iterate” back into “iterations”!

  • Truly iterate over the solution (holistic determination of where to take the product next) rather than just incrementing pre-determined backlog items
  • Deliver early and frequently, with very small (even daily) feedback loops – this makes us predictable

Create collaborative working agreements

  • Create flexible, collaborative working agreements with our customers which allow us to truly embrace change and deliver to customers’ present needs rather than their needs when we started
  • Allow customer to cut the cord early if they are happy with what they have (or not happy with progress)
  • Start from a position of trust rather than paranoia (which traditional contracts are based on)

Favour empiricism over guesswork

  • Keep work items small and simple, and limit WIP to create a predictable system
  • Slice features into simple, unambiguous stories using a heuristic rather than estimation rituals
  • Price work per feature if appropriate, using empirical average cost of features to guide price rather than a deterministic estimate of individual features
  • Use cycle time and throughput to make near-term prioritisation calls, not to determine release dates (there are no big releases in this approach anyway)

Shift focus away from estimation

  • Create a culture of honesty by removing negative estimation culture (i.e. get rid of story points and the notion of estimates as promises or deadlines)
  • Make work and project success about creative delivery of value (i.e. “what shall we do next?”) rather than “on time, on budget”, schedules, deadlines, etc.

2. Describe what you mean by a “slicing heuristic”

Essentially it’s a policy for how we break up our work. For example, “A user story must have only one acceptance test”. Rather than breaking features into stories and then estimating the stories, we can use the heuristic, measure our cycle times and then inspect and adapt the heuristic if required.

I’ve found the “1 acceptance test” heuristic to be consistently effective over different domains for creating an average story cycle time of 3 days or less.

3. How does your approach differ from that of Woody Zuill? Or, are there more similarities than differences?

I can’t speak for Woody but I feel that Woody’s approach is simpler than mine. He believes that if you follow the Agile Manifesto properly then the need for estimates dissipates.

I agree with him in principle but see systemic issues, particularly in analytic/mechanistic organisations, that I feel need to be addressed in order for #NoEstimates to strike a chord with more traditional managers and executives. At its core though, #NoEstimates is about exploring various approaches to delivering software without the use of estimates, and the commonality between our approaches seems to be the continuous delivery of small increments of high quality, valuable software.

4. Do you think any team can work without estimates? What’s the minimum “barrier to entry” ?

Any team (with the right coaching and knowledge) can embrace the slicing of work, limiting of WIP and measurement of throughput/cycle times, even if they are being asked to estimate with story points or time. #NoEstimates is not about refusing to estimate.

If you’re talking more about the overall approach from the portfolio level down, I’d say there is a minimum barrier to entry:

  • Fixed team (cost)
  • Allowance of variable (emerging) requirements/scope
  • Small batches of stories/features
  • Slicing heuristic to create roughly consistent actual/measured work unit size (“a few days”)
  • Incremental & iterative delivery capability of shippable code
  • Mini constraints such as weekly demo/review with customer (small, early and frequent releases)

This looks very much like any typical “Agile” team to me 🙂

5. What advantages does working without estimates provide your team over, say, a team that is using longer cadences, eg. Scrum?

My approach is entirely compatible with Scrum. In some ways I think that it’s what Scrum was intended to be (or at least, in my opinion, should be).

If a Scrum team is working in 2-week Sprints, truly iterating, delivering working software every Sprint, inspecting and adapting the product etc. then this looks very much like the approach I am advocating.

6. A common criticism of #NoEstimates is that when you slice off functionality to deliver (the “heuristic” approach) you are, in effect, estimating. Is this a correct interpretation? Why/why not?

Well arguably if you create a heuristic for creating “small” work then I can understand why it is interpreted that way. However, I don’t believe it is estimating. The point is to create simple and unambiguous story cards. The “smallness” is a by-product of doing this.

If we don’t get the smallness we’re looking for (after measuring the result) then we inspect and adapt the heuristic. At no point do we actually look at a card and say “I estimate that this is small”. We trust in the heuristic.

7. You’ve been a really vocal advocate for working without estimates, standing up to some tough questions from established agile practitioners. Why do you think this topic has so many people so roused?

Because the way software projects are typically governed is largely driven by estimates, so it touches almost everyone in the industry. It’s an established way of doing things so it is deemed controversial.

8. What would your advice be to a team considering working without estimates? What should their first steps be?

Don’t simply stop estimating. Try and get better at creating simple, unambiguous slices of functionality. Measure your throughput. Compare story count data with your story point data. Discover for yourselves if a #NoEstimates approach is right for you and a good fit for your organisational culture.

People Need Estimates

People need estimates. So they can predict how much software will cost and how long it will take.

People need umbrellas. So they don’t get wet when it rains.

Although, some people don’t need umbrellas. They have awesome waterproof jackets with hoods. They have solved the problem of “how do I stop getting wet?” with a different solution to the humble umbrella.

People need to know what time the trains are running so they can plan their trip to work. Some people do not need to know this because they take the London Underground, where trains typically arrive every 2 or 3 minutes.

What’s your point, Neil, you might be asking? My point is that when people are debating against the #NoEstimates movement, they always seem to gravitate toward the same two arguments:

  • People need estimates, so we should provide them
  • We cannot simply start building software without having an idea how long it will take or how much it will cost

To the first point, people only need estimates if we determine that the only solution to the problem of wanting to know “how long and how much” is to make a guess. People who have found other solutions to that problem do not need estimates.

I now wonder: just because the people who still need estimates have not discovered any alternative solutions, does that mean they need estimates or that they think they needthem? Or simply prefer to use them over other solutions?

People do not need umbrellas. They need a way to stay dry on a rainy day.

To the second point, I categorically want to put an end to the myth that #NoEstimates equates to #NoPrice or #NoDate. If you read my previous blog posts on the subject or read my tweets you will hopefully understand that my point is the absolute opposite. We DO need a price and/or a date. The only difference is how we arrive at those things.

With estimation, you guess one or both of them (and, in doing so, have a stab at scope too – otherwise what are you estimating?)

With #NoEstimates you set the price and/or date, either through experience and choice (for e.g. setting price/date for the kind of work you do regularly, with a fixed team and cost) or through a real budgetary or time constraint (e.g. “I’ve only got $100k, what can we build for that?” or “The Australian Open starts in 3 months so the Aus Open app needs to be ready to go live the day before”.)

You then incrementally and iteratively deliver, setting mini-constraints within the wider constraint that breed creativity, innovation and predictability of delivery, and have a flexible working and payment arrangement with the customer.

People need certainty about what they will get and how much they have to spend. Unfortunately there is no certainty in software design and development. However, I would argue that #NoEstimates gives greater certainty than estimating does.

When estimating a date or cost you are creating uncertainty around those things, because you are guessing. You are saying “we’ll deliver somewhere between here and here”. However, if your delivery date and/or cost is set by a real constraint, as advocated by the #NoEstimates approach, you have created certainty around those things.

Yes, you may decide to shift the date/cost as you get closer to the initial figures, or once the customer decides they are happy with what they have. You have been delivering frequently and learning about what you are building. You have been creating data, such as throughput and cycle times, and using heuristics and slicing to reduce work increment size, so informed decisions can be made along the way. But you will only go beyond those initial figures if the emergent value of what has been built, and other data you have gathered, suggests that you should. Scope remains uncertain whether you estimate or not.

People still need 500-page business requirement documents. People still need separate test teams and development teams. But there are alternative solutions which may render these needs unnecessary. The alternatives to estimation are real, both at the project and the portfolio level, and are being used by many people across the globe in varying sized businesses.

All I ask is that we consider those alternatives and do not stop searching due to need.

#NoEstimates Part 3 – The Palm Off

It is no secret to my Twitter followers, and perhaps beyond the Twitter-sphere, that I am on a crusade of sorts to get people considering other ways besides estimating when it comes to costing software development projects and tasks. Such a view remains controversial, even among Agile practitioners. People argue that there is no alternative; customers want estimates, so we must provide. Stakeholders need to know when things will get done. Estimation is seemingly one of the few remaining immutable practices hanging over from the Waterfall era.

One of the common criticisms of my view is that it is unduly dismissive. When asked by our boss or a customer for an estimate, we can’t simply palm them off and say “I don’t estimate! Talk to the hand, sir!”

Of course this is true. But I should point out that I actually see nothing wrong with being asked for an estimate of how long something will take. What I object to is being asked to carry out (or ask my team to carry out) estimation rituals whose results will then be used for making important business decisions.

We cannot palm people off, but what we can do is offer alternative, empirical approaches to traditional and “Agile” forms of estimating, explain exactly how we will provide the required information and why such approaches offer advantages over guessing “how long” or “how big”.

First off, I would suggest that there are many problems with the “how long/big” approach, the biggest of which is that such an estimate does not take into account the:

  • Inherent unpredictability of building software
  • Current work in progress (i.e. the team/dev may not be able to start the work “now”, or even for a few days, weeks or longer)
  • Capacity to do the work (i.e. the team/dev may make the estimate based on certain assumptions of team size which turn out to be false, or a colleague being there who ends up not being), nor
  • Any upcoming changes in priorities (i.e. something may jump above the piece of work in priority).

From a task point of view, what is estimated as a “10 minute job” may end up taking a day or longer due to one or more of the above. I’m sure you have seen this situation many times over. From a project point of view, this situation is magnified and can be hugely costly, even catastrophically so. 3 month projects become 6 months. 1 year projects become 3 years.

In a situation where there are small tasks flowing through from the customer to the development team that are unpredictable in their timing (e.g. BAU work queues, feature development, etc.), a far better, probabilistic approach to get some semblance of predictability is to do the following:

  • Measure actual lead times of every piece of work and plot them in a Lead Time Distribution graph
  • Measure throughput (you can start by simply counting the number of cards in the “done” column at the end of every week)
  • Use a fixed WIP limit on cards in progress (start, if you like, with the natural limit of team size)
  • You can now use Little’s Law to calculate average lead time for a card at position n in the queue, i.e. (WIP + n) / throughput:
    • e.g. Number of cards done in 1 week = 20, therefore throughput = 4 cards/day
    • Team size = 2, therefore WIP = 2
    • Lead time = (2+1)/4 = 0.75 days (i.e. on average it will take three quarters of a day for a card at the top of the queue to be delivered)

With the same formula you can predict where a card 2nd, 3rd or xth in the queue will get done, which is very helpful for guiding your prioritisation:

e.g. Using the same example above, a card 2nd in the queue will likely be done in 4/4 = 1 day, while a card 6th in the queue will likely be done in 8/4 = 2 days

Bear in mind the only way this formula can provide useful numbers is by having a WIP limit that is fixed (as far as possible). There will of course be variability in how long each card takes, but the law of large numbers will even this out to an acceptable average and it’s certainly far more scientific than asking people to estimate each card.

Note that if you use Scrum, and thus the team breaks down features into small tasks just-in-time at the beginning of every Sprint, you can use the same principles as above to determine when a new feature might be delivered (Scrum has a WIP limit over the Sprint length of the number of tasks in the Sprint Backlog, throughput is the number of “done” stories/tasks divided by the Sprint length, etc.).

Over time you can achieve a higher level of confidence with the predictions as you start to identify and split out different work types, determine probability of delivery times using your Lead Time Distribution graph, etc.

What about “how long will this project take?” !! Warning !! You can scale this approach up to the portfolio level. But… do bear in mind that building an entire software product rarely has a finite end point or a repeatable result because it is not possible (nor desirable) to define all of the scope required to deliver a delightful, valuable outcome. Use such predictions with extreme caution. There is no substitute in software product development for creating certainty around costs and delivery times via fixed agile teams delivering working software early and often, short feedback loops with the customer, etc.

So, next time you’re asked “how long” or “how big” about a software project or task, don’t palm off your boss or your customer with simply “I don’t estimate!”. Perhaps you might consider answering: “I don’t estimate! But… here is how we can save ourselves the cost of estimation meetings and make empirical predictions going forward to answer these questions with more confidence.”