#NoEstimates Part 3 – The Palm Off

It is no secret to my Twitter followers, and perhaps beyond the Twitter-sphere, that I am on a crusade of sorts to get people considering other ways besides estimating when it comes to costing software development projects and tasks. Such a view remains controversial, even among Agile practitioners. People argue that there is no alternative; customers want estimates, so we must provide. Stakeholders need to know when things will get done. Estimation is seemingly one of the few remaining immutable practices hanging over from the Waterfall era.

One of the common criticisms of my view is that it is unduly dismissive. When asked by our boss or a customer for an estimate, we can’t simply palm them off and say “I don’t estimate! Talk to the hand, sir!”

Of course this is true. But I should point out that I actually see nothing wrong with being asked for an estimate of how long something will take. What I object to is being asked to carry out (or ask my team to carry out) estimation rituals whose results will then be used for making important business decisions.

We cannot palm people off, but what we can do is offer alternative, empirical approaches to traditional and “Agile” forms of estimating, explain exactly how we will provide the required information and why such approaches offer advantages over guessing “how long” or “how big”.

First off, I would suggest that there are many problems with the “how long/big” approach, the biggest of which is that such an estimate does not take into account the:

  • Inherent unpredictability of building software
  • Current work in progress (i.e. the team/dev may not be able to start the work “now”, or even for a few days, weeks or longer)
  • Capacity to do the work (i.e. the team/dev may make the estimate based on certain assumptions of team size which turn out to be false, or a colleague being there who ends up not being), nor
  • Any upcoming changes in priorities (i.e. something may jump above the piece of work in priority).

From a task point of view, what is estimated as a “10 minute job” may end up taking a day or longer due to one or more of the above. I’m sure you have seen this situation many times over. From a project point of view, this situation is magnified and can be hugely costly, even catastrophically so. 3 month projects become 6 months. 1 year projects become 3 years.

In a situation where there are small tasks flowing through from the customer to the development team that are unpredictable in their timing (e.g. BAU work queues, feature development, etc.), a far better, probabilistic approach to get some semblance of predictability is to do the following:

  • Measure actual lead times of every piece of work and plot them in a Lead Time Distribution graph
  • Measure throughput (you can start by simply counting the number of cards in the “done” column at the end of every week)
  • Use a fixed WIP limit on cards in progress (start, if you like, with the natural limit of team size)
  • You can now use Little’s Law to calculate average lead time for a card at position n in the queue, i.e. (WIP + n) / throughput:
    • e.g. Number of cards done in 1 week = 20, therefore throughput = 4 cards/day
    • Team size = 2, therefore WIP = 2
    • Lead time = (2+1)/4 = 0.75 days (i.e. on average it will take three quarters of a day for a card at the top of the queue to be delivered)

With the same formula you can predict where a card 2nd, 3rd or xth in the queue will get done, which is very helpful for guiding your prioritisation:

e.g. Using the same example above, a card 2nd in the queue will likely be done in 4/4 = 1 day, while a card 6th in the queue will likely be done in 8/4 = 2 days

Bear in mind the only way this formula can provide useful numbers is by having a WIP limit that is fixed (as far as possible). There will of course be variability in how long each card takes, but the law of large numbers will even this out to an acceptable average and it’s certainly far more scientific than asking people to estimate each card.

Note that if you use Scrum, and thus the team breaks down features into small tasks just-in-time at the beginning of every Sprint, you can use the same principles as above to determine when a new feature might be delivered (Scrum has a WIP limit over the Sprint length of the number of tasks in the Sprint Backlog, throughput is the number of “done” stories/tasks divided by the Sprint length, etc.).

Over time you can achieve a higher level of confidence with the predictions as you start to identify and split out different work types, determine probability of delivery times using your Lead Time Distribution graph, etc.

What about “how long will this project take?” !! Warning !! You can scale this approach up to the portfolio level. But… do bear in mind that building an entire software product rarely has a finite end point or a repeatable result because it is not possible (nor desirable) to define all of the scope required to deliver a delightful, valuable outcome. Use such predictions with extreme caution. There is no substitute in software product development for creating certainty around costs and delivery times via fixed agile teams delivering working software early and often, short feedback loops with the customer, etc.

So, next time you’re asked “how long” or “how big” about a software project or task, don’t palm off your boss or your customer with simply “I don’t estimate!”. Perhaps you might consider answering: “I don’t estimate! But… here is how we can save ourselves the cost of estimation meetings and make empirical predictions going forward to answer these questions with more confidence.”

#NoEstimates Part 2 – Contract Negotiation and the Old Banger

This is the second in a series of blogs about why I believe we should not be estimating software projects. The first post talked about estimating at the team level, whereas here I talk about the contractual level and how to arrive at more Agile, iterative working arrangements.

Agile team, same old contract

Traditional software contracts, particularly with external parties, are based on:

  • Establishment of scope
  • Estimated time to deliver that scope
  • A price derived from that time + associated costs + profit margin

Many, if not most, of today’s software contracts are based on similar premises, even in supposedly “Agile” projects. In order to mitigate the risk of their deliverable running late and bumping up the cost, many customers demand fixed price contracts. Others demand that the supplier contractually fixes the delivery date to ensure meeting some obligation around the date and shy away from time-and-material engagements. Suppliers often like the fixed time approach as well because it creates predictability around cost. Fixed price contracts provide certainty around the project’s ROI, assuming it can be delivered at a low enough cost, and customers like to know how much they are spending.

There is nothing inherently wrong with any of these approaches or the reasons behind doing them. The problem lies in how we arrive at delivery dates and prices. In order for a contractual engagement between a supplier and customer to be worthwhile to the supplier it must deliver a positive return on investment. Usually this means that the money received from the customer for the supply of the product or service must exceed the money spent by the supplier providing it. So how do we balance that equation? Customers want certainty they will get what they want in the agreed timeframe and/or for the agreed price, while suppliers want to make sure they make a profit on the engagement. Seems simple enough. But what is missing from these scenarios? Even if both parties accept the well-understood iron triangle of time/cost, scope and quality, and that at least one of the three must be variable, is this enough on which to base a low risk and mutually valuable contract? I believe the answer is no, and not just because scope needs to be movable.

Quality is variable, not fixed

What?! Sounds controversial but I believe it to be true. In addition to the need for scope being variable, Agile folk also tend to talk about quality being fixed and uncompromising, meaning that time and cost can also be variable to deliver the best possible outcomes. Aside from the fact that leaving the cost and/or completion time of a project open is generally deemed an unacceptable way to conduct business, and likely why many businesses shy away from “Agile” contracts or working arrangements, I actually think it is un-Agile to fix quality. By this I’m not talking about code quality (the debate about what are bugs and acceptable levels of bugs in minimum viable and evolving products is for another blog post, another day). I mean quality in terms of what the customer defines as quality, and for me they are the only ones qualified to do so. IMO quality is an ever-changing variable in a project, just like scope. The difference is that the customer defines quality, either explicitly or implicitly, consciously or unconsciously. Scope, however, is defined by the supplier. Personally I think of quality in the context of products and services as:

“A subjective meeting of a need or requirement to the satisfaction or delight of the customer.”

If it is fair to say that what might delight a particular customer one day might not do so in 6 months time, and that what delights that customer right now may horrify another customer right now, I believe it is also fair to posit that quality ought not be fixed. I believe quality is what we should try and achieve, and it is what the customers want, but cannot fix what it means to achieve it. We will fail if we concentrate on time/cost and/or scope without making sure we are adjusting our delivery behaviour to suit the customer’s perception of quality. When we talk about projects being either “on track” or “off track” we always base it on our own interpretation of whether we are meeting the customer requirements. I believe theonly way we can know if we are on or off track is by asking the customer. They are the ones who know what they want. And this will most likely change. And this is fine! Great, in fact! That’s why we’re being Agile, and why they signed an Agile contract, right?

Don’t deliver the requirements, deliver what the customer wants

Delivering all the scope the customer wants may not actually delight them. It may even annoy them. Or cost them big time. They’ve hired you because you’re an awesome web design company with a great track record. They love your previous creative, innovative designs. And now you have done exactly what your customer has told you to do and it looks crap because your customer does not have a flair for web design. They are the customer, you are the supplier. You are the expert in what you do. You should be telling the customer the scope that will meet their requirement, not the other way round. And they should be telling you whether you are meeting their requirements or not. I believe you can never be “on track” in a truly Agile project, at least in a Gantt chart or velocity-based-Agile-release-plan sense, because the entire fabric of what you are building can change at any moment. If the contractual arrangement is done right then change is absolutely fine, to be expected and welcomed.

Agile contracts – the reality

Breach-of-ContractSo what really is an Agile contract?

Fixed price contracts are fine. Fixed time contracts are fine. But here are the caveats:

  • Do not fix time based on an estimate of cost because that inherently means you are agreeing to up-front scope detail that will likely bite you on the arse later and restrict the customer’s ability to request changes (and yours to welcome them) for their competitive advantage
  • If the customer does not fully understand and embrace the inherent unpredictable, creative and innovative nature of quality software solutions then work with them at your peril
  • If you don’t want to turn away work so you try and agree scope with the customer because “they insist”, and then base dates and times on estimates, do not pretend this is an Agile contract and make sure all parties understand the implications of this
  • Know your costs by having a fixed team and determine a “final” delivery date, or allow the customer to determine it
  • If the delivery date is acceptable to both supplier and customer then you now have a certain delivery date, no guesswork required; if the customer wants delivery sooner, reduce the price AND the expectation of quality
  • When you purchase something more cheaply outside of software, e.g. a cheap old banger of a car, you can assume you will likely receive a lower level of quality – why is software any different?
  • Negotiate a flexible, iterative, drip-funded contract that allows the customer to retreat early (either because they’re already happy with their product or because they’re not happy with the progress; if it’s the latter learn from their feedback, improve and move on)
  • The aim is to delight the customer and make a profit so do not simply do what they ask you to do; they are buying your expertise and guidance for meeting their need, so don’t take this responsibility lightly and think you’re serving the customer simply by “delivering customer requirements”
  • Deliver early and often (duh!); iterate, don’t just increment, and make this part of the working agreement
  • If possible give the customer a sense of the kind of outcome they can expect for varying price and/or delivery times (based on previous work done by your company) and given them options to “upgrade” or “downgrade”

99 seconds

Remember we’re supposed to “welcome” change?

Yes, don’t try and fix scope. But be prepared to move around on quality also. Allow the customer to accept an earlier version of your product because it does the job and they’re delighted they don’t need to spend any more cash on achieving their desired outcome. Or to love their product so much that they now want to spend more enhancing it. This is variable quality, in my book. Variable scope refers to the cost-side of building software; the amount of work we need to do to reach a specified outcome. Variable quality refers to the value the customer feels they are getting. It’s subjective, dependent on the customer and their particular circumstances. Delivering high value outcomes to the customer may cost more than lower value outcomes or they may not, depending on what the customer feels about the iterative outcomes. That “old banger” that you bought for $1000 may actually provide very high value and quality to you personally. Or it may be housing a classic engine that you didn’t previously know about, giving it emergent value. To someone else it’s a worthless piece of junk.

111610_0208_MicrosoftWo3In the same way software solutions, products and services are entirely subjective in their quality. Some people think Microsoft Word is awesome and feature-packed and they base their entire business operations around it. Some think it is terrible, buggy and doesn’t do anything they want it to do. Let’s not pretend that delivering “quality” software is a predictable outcome any more than fixed scope is.

Variable quality pertains to the wonderful opportunities we ought to have with Agile software development for correcting the course and building the right thing; truly welcoming and embracing change for the customer’s (and our) benefit. This is what Agile contracts should be about IMO. Remove the uncertainty of time and cost by making them certain, and celebrate with your customers or suppliers the uncertainty around exactly what will be built. Why not consider basing your contracts on a mantra more along the lines of:

“We guarantee we will work with our customers’ time and budget constraints to iteratively build and evolve a delightful outcome to an agreed level of expectation?”

And for everyone’s sake, we should not be estimating in order to do it.