False Gods, Silver Bullets and the Myth of Innovation

I am privileged to publish this guest post from the brilliant and lovely Michael Rembach (@mrembach).

In October I stumbled across a blog article about product development using Scrum and the hindering effect that Scrum can have on the innovation process especially if the organisation is fully ‘agile immersed’.  The blog was written by Brian de Haaf (@bdehaaff) who is the co-founder of Aha! – A product management software company.  While the article was well written and brought up many salient points about innovation, I disagree with the overall premise that Scrum may have innovation-limiting behaviours  You can read the original article here zite.to/17HnE4S .

The first thing I’d like to point out is that I agree with the points about innovation in the article.  Innovation practices, such as having a shared vision, engendering trust in your organisation and having a strategic direction are all vital ingredients for success and even more so in technology companies.  The thing about innovation is that it’s a cultural thing and no framework/methodology/philosophy in the world is going to make your company innovative without the desire (or need) to. Having a myopic view of your product because you’re ‘Agile’ misses the point of the delivery focus and discounts the innovation-enabling practices that Agile encourages.

Scrum, and other Agile methodologies, are essentially delivery focussed which is why there is a requirement for product owners to focus strongly on the Sprint cycle and the short-term delivery timeline that it brings.  However, this does not and should not excuse the product owner for not checking that what is being delivered is aligned to the strategic goals for the product or in fact, the organisation.  The two aren’t mutually exclusive and a product owner is responsible for communicating that vision to the project team so that they are aware of the purpose of the product.  Constantly checking in with the vision by all the team should ensure that what is being built doesn’t deviate from the intention of the product’s purpose.  The product owner is simply not performing her role properly if she suffers from the myopic concern with delivery-cycles without also ensuring that the product is meeting its intended strategic objectives.

Rather than inhibiting innovation, I posit that Agile has a number of practices that encourage innovative behaviour:

    1. MVP – the primary reason for creating a minimum viable product is to determine that what you’re trying to produce is viable, but it also serves a couple of other important purposes.  The first is prototyping; where you have the opportunity to experiment with your solution, try something small and novel and see if it works and the second; it gives you the opportunity to solicit feedback from your clients, the product ecosystem and anywhere else.  This is a primary source of knowledge for decision-making.
    2. Fast-failure – Agile methodologies allow you to fail quickly and learn some valuable lessons before it costs you too much.  Innovation is all about finding out new ways to do things and failing fast and safely is one of the best ways to forge new paths.
    3. Continuous learning through retrospectives – a learning organisation is an innovative organisation and retrospectives provide an excellent opportunity to improve not only what we are producing (again, you can look at the strategic alignment at the end of every sprint or release cycle), but also how we work together.
    4. Embracing change – if making changes to your product is painful then your ability to be innovative will be too.  Agile methodologies accept that change is inevitable from the get go and therefore provide less resistance to innovating during the development of a product.

Innovation is difficult at the best of times.  As Clayton Christensen illustrates in his famous Innovator’s dilemma, history is filled with the burnt out shell of successful companies that died as a result of not being able to change.  To succeed, innovation needs to be part of the organisations culture.  The premise that progressive change-embracing frameworks like Scrum inhibit innovation does not recognise these aforementioned practices.  Agile won’t make you innovative, but it sure can help encourage it.

#NoEstimates puzzle experiment in Melbourne

Craig Brown and I ran a variant of Chris Chapman’s now famous jigsaw puzzle experiment at a meetup of the Melbourne Agile and Scrum User Group on Wednesday evening.

Everyone had fun and was intensely engaged throughout. There were loads of interesting dynamics emerging from the teams, perhaps surprising given the contrived nature of the experiment.

Set up

  • We set up three same-sized (10-12 people) teams, each with:
    • an identical jigsaw puzzle (way too big to be completed)
    • a Product Owner (to provide the vision and direction) and
    • a Scrum Master (to help the team achieve the PO’s vision)
  • We opted for 3 * 15-minute iterations, with 3 minutes for a Retro in between
  • Each team was told to use a different method – one was a Scrum team, one was a “mob team” and one was a “no rules” team. Here’s what that meant:

Scrum team

  • Must have Planning (including estimation), Review and Retro in each iteration
    • We provided Planning Poker cards for the estimation but the team was free to choose whatever estimation method they liked
  • Must only work on “stories” agreed in Planning – new stories can’t be introduced mid-iteration
  • Stories are only “done” when PO accepts them (in Review or before)

“Mob” team

  • No formal ceremonies required
  • Team all works on one story at a time until “done” (single-piece flow approach)
  • No estimation
  • Retro encouraged but not “enforced”

“No Rules” team

  • Can work like the Scrum team, the Mob team, any combination of the two, or any other way they like

Outcome

  • 600_301537052600_301537062Scrum team delivered most stories (3; the other teams delivered 2 each)
  • Whole group was asked to vote on which they thought was the best outcome
    • “No rules” team won (emphatically)
    • Scrum team lost

Interesting Observations

Here are some empirical observations of the evening’s events and outcomes, along with my interpretation of what they indicate in an Agile/#NoEstimates context (==> in bold italics underneath the observation).

Scrum team

  • Delivered most in terms of stories but least in terms of value, both for their Product Owner and as voted for by the wider group
    ==> Output <> Value
    ==> Comparing teams in a useful way would require consistent measures of both effort and value velocity across teams
  • Spent far too large a proportion of time (particularly the first iteration) in planning, and needed to be alerted to this fact
    ==> Consistent timeboxing is important to ensure there is time to do all that is required, and for less variability of outcomes
  • A member of the team openly admitted that he inflated an estimate because he did not agree with the value of the story that the PO wanted to do next
    ==> Estimates are often gamed, and for various reasons

“No rules” team

  • Implicitly chose not to estimate, but instead to maximise the time they had for building
  • Eventually delighted their Product Owner (and wider group), but during the game the PO felt like:
    • The approach to delivery was too ad-hoc, even chaotic, especially at the beginning
      ==> Teams must collaborate in order to be co-ordinated, improve and deliver the right outcomes
    • Stories were too large (epic) so delivery all happened near the end rather than incrementally
      ==> Smaller stories have lower variability and can help with early and frequent delivery, creating better predictability for PO/customer and lessening the need for estimates
      ==> Larger, higher variability stories rely on estimates of time, or at least relative size, to provide the illusion of predictability
  • Started with no process at all but this was deemed unproductive (with such a big team), so they split into smaller teams with focused goals
    ==> Smaller teams are more effective because it is easier to collaborate, change direction, gain consensus, etc.

General

  • Scrum and Mob team both delivered purely incrementally (concentrating on edges) rather than iteratively (identifying a recognisable area of interest and building upon it), although stories were clearly too big
    ==> An iterative approach is crucial for risk management, predictability and delivering the right thing (value), i.e. without such an approach you have no choice but to estimate
  • Product Owners all felt like they weren’t being listened to – this had particularly bad consequences for the Scrum and Mob teams, perhaps due to their purely incremental approach
    ==> Important for all team voices to be heard, especially given the PO is driving what should be built in order to deliver on the vision

Stand Up and Shut Up

As with many simple and now commonplace “Agile practices”, debates still rage on about the Daily Standup (Scrum) meeting, a meeting which has somehow become a ritualistic signal that a team is “Agile” but is often an equally conspicuous signal of the exact opposite.

I’ve been in many organisations where God forbid anyone asks whether we should get rid of the meeting, or even change it, despite the fact that no one is getting any value out of it every single goddamn day*.

*Except some managers. A daily status update meeting? Terrific! The Daily Standup is an opportunity to micro-manage people every single day without having to approach their desks!

I digress. The point is, people still question the value of the Daily Standup and, if it is indeed valuable, how we might make it more effective.

I share the view of the Scrum Guide on this – at least in what the spirit of an effective Daily Standup meeting is, if not necessarily the prescribed format.

An effective Daily Standup meeting, for me, is one in which the team inspects and adapts both product and process.

That is to say it is an alignment meeting. A daily planning meeting. An opportunity to change our path if there is a better one. We do not have to (and should not) wait for the Sprint Review (product) and Retrospective (process) for this. Continuous improvement is about daily inspection and adaptation.

Here are some of the more effective questions that can be used in a Daily Standup meeting:

  • How will we work together today to move toward our goal?
  • What should we focus on today?
  • What should we not do that we originally thought we would do?
  • How will we remove this impediment right now?
  • Given we are a little behind, how might we simplify this product increment?

It is about purposeful intent for the day. It is certainly not intended as a status meeting. If managers and others outside of the core team are not getting the information they require from conversations or the team wall then it will surely pay dividends to improve visibility and transparency in the way people interact while doing their work rather than have a daily status update meeting.

In fact, I would go as far as saying that the ritual of an unchanging Daily Standup meeting is usually a smell of poor collaboration in and between teams on the actual work to be done. Some companies mistake this meeting as a way of actually getting people to collaborate. It’s almost as if they think that the benefits of collaboration, as Agile promotes, can be gleaned simply by having this meeting.

standup-construction1Unfortunately it is not that simple. Standing (or sitting) people together does not make them collaborate.

Collaboration is an organic thing and only comes if the “way the work works” is designed to encourage it.

I sometimes see or hear the argument that, “because we’re Agile we should make the meeting fit with the way we currently work“, and that doing this will intrinsically make it more valuable. So, the argument continues, it’s OK if it becomes a status update meeting because that’s what the environment demands.

The issue with this approach is that the environment in which you currently operate is likely one of managers wanting status updates. One of traditional ways of doing things.

But in order to be effective with an Agile approach we have to do things differently. To think differently.

Agile does not mean “make compromises”. It is about mindful changes in the way we work to move toward improved effectiveness. If something feels a bit different and uncomfortable then it may well be a sign you are on the right track.

As coaches, we ought to let the team decide how they can get most value from a Daily Standup meeting. Then, rather than focusing all our attention on how to improve the meeting, we should instead be helping the managers create an environment in which actual collaboration (working together effectively toward common goals) is encouraged and starts to feel natural.

Where excellence, rather than dogma, can prevail.

P.S. Standing up is not mandatory! If the meeting is timeboxed to 15 minutes then it will be quick regardless of whether you’re sitting down, standing up or doing the cha-cha.

Revisiting the Three Amigos

Next week I am speaking at a SIGiST (Specialist Group in Software Testing) event in Melbourne. Having to prepare my presentation has encouraged (OK, forced) me over the past couple of weeks to re-immerse myself in the world of quality, testing and BDD (Behaviour Driven Development).

Despite everything we’ve learned about the value of conversations when deciding what to build into our software — about the value of automating as much of our testing as possible in order to shorten the feedback loops between things breaking and us knowing about them breaking, and to instill confidence among the stakeholders and the team that we can rapidly add new features without breaking existing ones; about the value of taking a test driven approach to building our software, based on real user behaviour rather than code behaviour, to enforce good design practices and ensure the software does what it is supposed to do — I still constantly see and hear of teams struggling with their approach to quality.

Some are struggling to find time to improve due to a combination of legacy systems with brittle or no automated test coverage and looming deadlines for new products or features. Some are struggling to create a short enough feedback loop for testing software increments as they are built so that problems can be addressed before code is deployed, or before developers have moved on to the next or even the next feature.

There is no denying that it is crucial to get the technical practices right from the start. Enough has been written about this. BDD at all layers, continuous integration and automated acceptance and regression tests.

However, when you find yourself in a situation where you are adopting a legacy system or process – i.e. you or your predecessors haven’t got your technical practices right from the start – then your only viable option will usually be to improve things gradually. Have developers learn how to and implement automated acceptance tests. Chuck out and replace flaky record-and-play UI tests with robust unit, integration and browser tests using best-of-breed tools. Embed testers in the development team. Gradually start to do all the things that ideally would have been done from the start.

It seems like a desperate situation, but all is not lost. Far from it. I feel that a common mistake teams and businesses make is to place too much focus too early on the necessary technical improvements.

In my experience, the most important thing to improve is the conversations between the business people, customers and the development team.

One effective technique for doing this is The Three Amigos approach, where the customer / Product Owner / BA has a chat with a developer and tester from the team to agree on the acceptance criteria for a new feature or story before it is undertaken. From this conversation the team can decide exactly what tests are needed, and where they should be implemented, in order to prove that the completed functionality will do what is supposed to do.

A mature Agile team would now write the necessary tests in their tool of choice (e.g. JBehave for Java), the developers would write just enough code for the tests to pass, then refactor. When all the acceptance tests pass, the story is considered “done” from a functional perspective.

But what if the tester and/or developers have little or no experience with an automated testing approach? I have worked with teams in this situation and it cannot be fixed right away (or even at all if there is no willingness from the business to invest in training and slack time to address the problem).

Let’s say the tester is traditional in his approach, and would typically create test cases which he will use to manually test the code when it comes to him from the developer. What tends to happen here is that the developer writes the code for the story, then hands it off to the tester, who then hands it back because the code doesn’t do what the tester expects it to do. This to-ing and fro-ing can happen once, twice, three times. It’s time consuming and frustrating for everyone, and makes it very difficult to complete product increments in a timely fashion.

Transform-through-conversations-photoHowever, if the tester and the developer have a conversation before the developer starts coding (with the PO/BA in the Three Amigos meeting, or just-in-time in a story kick-off), the tester can take the developer through his test cases (derived from the acceptance criteria) so that the developer understands everything that the tester expects to work when he is handed the code.

Over time in these conversations the developer will start making suggestions, so the test cases become more collaborative and thus effective. He will also want to make sure the story does not bounce back to him from the tester when he’s coded it, so he may do some more manual testing of the functionality or even write some (more) unit tests before handing the story to the tester. His confidence in his code is likely to have improved, and the bounce-backs become the exception rather than the rule.

The key to building in quality is first and foremost in the conversations because they create improvements in the way we work together, whatever situation we are in technically. The good technical practices will emerge from the better conversations. Agile is largely about focusing on technical excellence but, as the first line in the Manifesto tells us, more important is the interactions between the people doing the work. Continuous improvement allows us to start where we are and take one step at a time.

These up front and ongoing conversations, such as the Three Amigos, can have a massive impact on your effectiveness both individually and as a team, and on the quality and maintainability of your product, increasing your agility to adapt and innovate . Adding such conversations to your process is a great sign of continuous improvement and embracing the first and most important line of the Agile Manifesto.

Scrum Basics Part 2 – Monitoring Progress Toward a Goal

"Various projective practices upon trending have been used to forecast progress, like burndowns, burn-ups, or cumulative flows. These have proven useful. However, these do not replace the importance of empiricism. In complex environments, what will happen is unknown. Only what has happened may be used for forward-looking decision-making."

-- Scrum Guide

Agile/Scrum teams are often asked to estimate how long a release might take. Or an entire project. Sometimes this is done under the guise of relative size estimates like T-shirt sizes – or, perhaps more commonly, story points – coupled with an estimated (or guessed) velocity. This is sometimes done even with new teams that have no velocity history.

Scrum, as defined in the Scrum Guide, places a large emphasis on the use of empiricism. Aside from the quote above, the following nuggets can also be found:

"Scrum is founded on empirical process control theory, or empiricism. Empiricism asserts that knowledge comes from experience and making decisions based on what is known. Scrum employs an iterative, incremental approach to optimize predictability and control risk."
"[The Scrum Master helps the Product Owner with] Understanding product planning in an empirical environment"

My interpretation of Scrum is that, while the Development Team are expected to estimate each PBI (Product Backlog Item), they are not asked nor expected to determine delivery dates, or how much work will be completed by a delivery date.

At Sprint Review:

"The Product Owner discusses the Product Backlog as it stands. He or she projects likely completion dates based on progress to date (if needed)"

So, the Product Owner uses the estimates on the PBIs combined with the empirical knowledge gained from what has actually been done to determine completion dates of a set of PBIs (e.g. a release). At no point does the Product Owner ask the team what will get done (beyond the current Sprint).

This use of empiricism is often neglected by Scrum teams. Teams are asked to project release dates, sometimes several months out, without any velocity history. This is not making projections based on what has actually happened. It is not empirical, and does not work in a complex, ever changing environment.

"A Product Backlog is never complete. The earliest development of it only lays out the initially known and best-understood requirements. The Product Backlog evolves as the product and the environment in which it will be used evolves. The Product Backlog is dynamic; it constantly changes to identify what the product needs to be appropriate, competitive, and useful. As long as a product exists, its Product Backlog also exists."

If you are using estimates, it is important that you use probabilistic estimates based on real, empirical data. Scrum suggests this. Practitioners suggest this also. Don’t ask the team to forecast any further out than the current Sprint. As the Product Owner, use real data to make forecasts and decisions. Asking the team to make longer term projections is not respecting the data showing what is actually getting done.

Monitor progress, don’t try and predict it.

Systems thinking and the naughty dogs

Envisage this situation. I go to bed and forget to let the dogs out. When I wake in the morning, and go into the living room, the dogs have crapped on the rug.

Who is to blame?

My first reaction is likely to be to blame the dogs. “Sammy! Jake! You dirty dogs!”

My wife will likely blame me (once she finds out I didn’t let Sammy and Jake out).

What’s next?

Well, I don’t want that happening again. How can I make sure I don’t forget to let the dogs out again? Another foul up (forgive the pun) will be difficult to take.

Perhaps I could put a sign up on the wall in the landing, on the way to my bedroom: “DON’T FORGET TO LET THE DOGS OUT!” Won’t be foolproof, but it might help. My wife might decide she can’t trust me to let the dogs out every evening, so she will start reminding me every night, or coming into the living room to check.

Of course she might forget to do this one night. If that happens to coincide with a night on which I also forget, the same outcome may occur.

Now who’s to blame?

This kind of scenario might sound oddly familiar if you work in an IT department or work for a software development company. An innocent mistake (like releasing an obscure but potentially damaging bug), leading to blame of the individual, leading to more control of releases (processes and procedures) and a “don’t fuck up” culture.

Of course we don’t want the dogs to crap on the rug. Blaming me for this incident, imposing more control (the sign on the wall) and reducing trust in me (my wife checking I’ve put the dogs out) *may* solve the problem. But in reality there is still a chance that it will happen again. People make mistakes. People repeat mistakes.

Infant-2-and-doggy-doorProblem dissolution

By employing a systems thinking approach to this scenario, we can look to *dissolve* the problem. That is, the problem of “the dogs might crap on the rug during the night” is actually removed rather than its probability reduced.

If I install a doggy door, the dogs can get in and out whenever they need to, so they will never be stuck inside when they need to crap. My wife will never have to worry about me messing up again, and blaming me for my stupidity. We won’t need signs up on the wall, serving as a constant reminder to myself and my family that I messed up.

Sometimes buggy software will be released, no matter how high the quality of our code or the stringency of our release procedures. Because people miss things. People make mistakes. People repeat mistakes.

If we make releasing really quick and easy, we can update our tests and release bug fixes before there is any time for blame and increased control to become necessary.

Do you look to merely solve problems in your organisation, or to dissolve them?

Scrum Basics Part 1 – Activities, not Roles

This is the first in a series of small posts aimed at new Scrum teams, organisations newly adopting Scrum and people who have been doing Scrum for a while but are struggling to get the results they crave.

This post is based on a response I gave to a question in a LinkedIn forum:

“The BA role is an integral and implicit part of Product Owner Role in Scrum. What is your take on this?”

This is a very common question among those new to Scrum and Agile. It’s an interesting one and a classic example of why, in my opinion, companies the world over are failing to do well with Scrum.

To begin to answer it, I will let the Scrum Guide do the talking:

  • The Scrum Team consists of a Product Owner, the Development Team, and a Scrum Master.
  • Scrum Teams are self-organizing and cross-functional.
  • The Product Owner is the sole person responsible for managing the Product Backlog.
  • The Product Owner is one person, not a committee.
  • Development Teams are cross-functional, with all of the skills as a team necessary to create a product Increment;
  • Scrum recognizes no titles for Development Team members other than Developer, regardless of the work being performed by the person; there are no exceptions to this rule;
  • Scrum recognizes no sub-teams in the Development Team, regardless of particular domains that need to be addressed like testing or business analysis; there are no exceptions to this rule; and,
  • Individual Development Team members may have specialized skills and areas of focus, but accountability belongs to the Development Team as a whole.

Departmental silos are entrenched in the way companies typically do things. They are part of the system. The culture. As a result, the urge to maintain departmental silos is strong.

I would suggest this is a key reason why Scrum implementations might (and do) fail.

Straight off the bat, certain elements of the Scrum Guide are typically ignored or deliberately rejected. These elements may or may not turn out to be key in your organisation, but the fact is they are in there for very good reason. It is a mistake to assume from the outset that your context requires removal of these elements.

Scrum is not asking companies to remove departmental silos, but it is asking that these silos are ignored such that they do not exist within the Scrum team. In the Scrum team, everyone building the product increment is part of the Development team. There are only 2 other people in the team – the Product Owner and the Scrum Master. That’s it. That’s the Scrum team model. Period.

There is absolutely no prescription as to who should be in the Development Team, only that the team has all of the skills and capabilities required within it to build a product increment, and that the team jointly owns all of the work, activities and decisions. In order for effective teamwork to flourish, Scrum says that roles should be left at the door.

That does not mean that our individual expertise and experience is left at the door along with our job titles. On the contrary, the best self-organising teams decide how best to leverage the expertise within the team.

If the question asked in the LinkedIn discussion was actually:

“Are the typical activities undertaken as a BA part of the Product Owner’s responsibilities in Scrum?”

then my answer would be that these, and any other activities involved in building and managing a product’s development lifecycle end-to-end, are shared between the Scrum Master, Product Owner and Development Team. This is made very clear in the Scrum Guide.

To that end, there is no “BA role” in Scrum, much like there is no “tester”, “QA” or “UX designer” role. Roles are part of traditional siloed thinking. Scrum (and Agile) focus (deliberately and alternatively) on cross-functional teams. Roles are a function of the particular company, not the activities that need to be done as part of product development.

To get the best results from Scrum it is a good idea to stop thinking about what roles you need in the team, and instead think about what activities are required to build your product. A good self-organising Scrum team will share these activities regardless of whether they have a specialist, designated BA or not.

Personally I like to encourage “collaborative analysis”, where all of the “what” and “why” for every decision, every story, is talked about by the whole Scrum team. Then the “how” is handled by the Development Team.

The popular model of having BAs “writing stories” and handing them off to the developers in the team is highly ineffective, not the hallmarks of a collaborative, self-organising team and about as far from both Scrum and Agile as you can get.

To build products effectively with Scrum, it’s a good idea to map out all of the activities that are required to build the product. Forget current roles and responsibilities for now. Once you’ve listed the activities, gather a team that can execute those activities in their entirety. If your company has BAs and you need one of them for your Scrum team then by all means have them in the team.

But please remember to ask yourself this key question:

“Is the BA part of the Development Team or are they the Product Owner?”

In Scrum, they can’t be both. And they can’t be neither.

Stop using time in relative size estimates!

The SAFe approach to normalised story points makes a classic mistake that everyone seems to make with story points. It is not “relative sizing” to compare stories to a reference story that has been estimated in time (in this case “about a day”).

As soon as you introduce time as a basis for your reference story, and use e.g. story points with Fibonacci sequence, all of the comparisons you make are based on time, i.e. a 2 point story pertains to 2 days, 5 points to 5 days, etc.

Even if you are not doing this consciously you will do it unconsciously. So all you have done is estimated “how long” the stories will take to deliver. This is not relative sizing!

The whole point of using relative sizing instead of time-based estimation is that humans are better at comparing the size of things than we are about making absolute judgement of size, e.g. we’re good at being right that building A is bigger than building B, but we’re not so good at being right that building A is about 200 metres high and building B is 150 metres.

Unfortunately when it comes to tasks that we perform, our natural tendency is to use absolute terms because the “size” of a task essentially equates in our brains to “how long”. The fact that story points are numbers doesn’t help with this. Where story points completely lose their value is when we start deliberately equating a point value with a length of time.

True relative sizing of a backlog is to pick a low value story (one that you are unlikely to implement for some time) and do not estimate it at all. What you now do is compare other stories to that story, i.e. I think story C will take longer than story B, story D will take longer than story C, story D is about the same size as story C, etc.  At no point do we actually predict how long something will take. We are simply saying which stories will take longer than others, by our estimation.

When a new story emerges you then do the same thing – decide if it will take longer than the reference story (which, because you have not yet implemented it, you will not be influenced by the actual time it took), less time or about the same.

You can now measure progress against the total backlog as you deliver the stories.

One thing I do agree with in the SAFe approach is that you should not do any re-calibration/estimation. As soon as you start re-estimating stories based on how long things are actually taking you are being influenced by time. This can not only throw off the relative calibration of the backlog but also ignores the inherent variability of software increments; i.e. there will be outliers within size groups that take significantly longer (or shorter) than the modal average.

P.S. If you’ve read my other #NoEstimates stuff on this blog you will know I do not advocate the use of story point estimations at all, especially due to the way they are typically misused and abused. However, there may be some potential value in doing relative size estimates (e.g. T-shirt sizes), if done right, for one or more teams working from the same initial product backlog in order to give some indication of the overall viability of the initiative and to provoke discussion within the team(s) about the value and possible approaches for undertaking individual pieces of work, aka “what shall we do next”.

#NoEstimates – An Alternative Means of Risk Management

Introduction

A continuing theme of counter-arguments posed at the #NoEstimates ideas is that development cost estimates are required in order both to manage risk and to derive value.

This blog post intends to give further insights into how risk can be effectively managed, and how we might determine the value of our initiatives, without the need for making up front and deterministic development cost estimates.

Risk

“Risk is the probability of an unfavorable impact to the project” – Glen Alleman (@galleman).

From the risk angle, the argument goes along the lines that the built-in “risk management” in Agile approaches is not aligned with conventional definitions of risk management in software development.

I’ll go along with this. Agile (and #NoEstimates) does not take the conventional approach to software risk management, which sees project success as “on time, on budget” and thus requires an up front estimate of total scope, cost and duration.

Agile/#NoEstimates offers an alternative way to manage risk on projects (and, no, I’m not taking about Agile Estimation, the spin-off brand of traditional estimation promoted by Mike Cohn). I’ll explain more about this later.

Value

The argument regarding value is that estimated cost is required to determine value, given that value is related both to the timing of when things are released and how much it costs to develop the things that will (potentially) generate value. That the worth of something to someone can only be evaluated if we know how much that things costs.

Again I agree to an extent, but there are two key sticking points for me here. One is that we only know how much software development costs after the fact. People say “we need to estimate because we need to know the cost”. Estimating, however accurately we think it is being done, does not allow us to know the cost.

Before the event we can only estimate what will be done and how much it will cost. In addition, the further out we are estimating cost and value, the riskier (and potentially costlier) our estimates become.

By estimating, rather than fixing, cost we have no greater insight into the value, which is also estimated. Essentially we are increasing our risk by estimating both cost and value rather than just value, which is what #NoEstimates promotes. More on this later.

The other sticking point is that value is often highly subjective and personal. I know how valuable a particular brand new Ferrari is, partly because I know how much it costs. That said, if you gave me two different Ferraris to test drive and didn’t tell me how much they cost, I would tell you which one I prefer. Which one was more valuable to me. This has nothing to do with the cost. The one I prefer might be significantly cheaper, but its value to me is higher because it’s more fun to drive and I prefer the look of it.

The same applies with software. There is so much to consider when we try and measure value. Aside from the empirical measure of monetary returns, we have to consider the needs of the customers, the stakeholders and our corporate strategy (to name but a few), not to mention the fact that all of these things change over time.

Agile is about delivering value early, not trying to predict how to maximise value over a given timeframe or a product’s lifecycle. It is the early delivery of value that allows us to tune and adjust our course for maximum longer term benefit.

This is why it is an alternative, and completely viable, approach and should be considered as such.

Agile Risk Management

The key aspects of Agile that help us manage risk effectively are:

  • Iteration
  • Continuous selection of highest value work (i.e. making decisions)
  • Fixed, cross-functional teams with 100% focus on current project
  • Early and frequent delivery of end-to-end working software increments and
  • Empirical measures of progress toward goals

Waterfall-computer-wallpaperWith Waterfall projects, the need for conventional risk management is clear. We have no way of measuring progress from day one in terms of working software because we are carrying out requirements analysis, specification and design phases before we write a line of code. People are often working on multiple projects and so we must allocate a percentage of their time to the project at hand.

The only way to measure percentage progress toward project completion is to have a breakdown of the SDLC phases and tasks within each, estimated in days/weeks, and tick them off as we go along. If we don’t complete all the necessary tasks for a given phase in the estimated timeframes, we are “off track” and we need to take corrective action.

With a phased delivery approach, the only way to manage risk is to have an estimate of the total scope, cost and duration of the project.

But if we are working in an Agile way, we are not taking a phased approach to project delivery. We are delivering full end-to-end working solutions in an iterative manner, early and frequently. We are working in fixed, cross-functional teams so teams costs are known and consistent.

This approach allows us to manage risk and measure progress toward project completion (meeting of stakeholder goals within a given budget) from the get-go.

Progress

If we are truly iterating by delivering vertical slices through the system, after our first iteration we will be able to measure progress toward the project goals. We will have delivered a working, albeit perhaps low quality, solution to the problem. We may even have actually met the project goals.

Either way, we can inspect what we have done and decide if we are on the right track. If we are, we can iterate over our solution, improving quality in the desired areas and incrementing new features. If we are not, or we see a better way of solving the problem, we can throw away what we’ve done and start again. We may even decide to scale up our efforts and add more teams, if there is emergent estimated value in doing so.

Given in Agile we are delivering end-to-end working software from the get-go, we are not burdened with the problems we faced in our Waterfall projects for measuring progress. We have the ability to empirically measure progress because we are delivering “done” functionality, as opposed to hitting pre-determined “milestones” which are not based on what we have actually delivered in terms of a working product.

green-traffic-lightIn Waterfall, so long as we are hitting our milestones then the project status is “green”. For software product development projects, this means that we are deferring our risk management until we actually start writing code. We don’t know that the scope of what we want to build is achievable, and we can’t reduce scope until we actually realise it’s too much (well into the development phase, deep into the project).

In Agile we can manage scope right from the beginning, because we are continually focusing on building the most valuable thin, vertical slices which represent iterations over an end-to-end solution to the problem. We can empirically measure how much we got done and how much is left to do. We can regularly take proactive decisions to cut scope or switch to an alternative approach to improve our chances of delivering a successful outcome. What should we do next for maximum value and maximum impact in meeting our goals? What should we not do? What is the simplest approach for our next iteration?

This is risk management.

These kinds of conversations enable us to focus on doing the simplest thing, for maximum impact, given the budget that we have available. To not wait 9 months to deliver a solution but to deliver a solution in 1 month, then make it better.

mona_jonah-640x209Most “Agile” projects are not managing risk

If we decide up front in a project inception on the requirements (product backlog) and solution we will be sticking to, and estimate it will take, say, 9 months, all we will do is incrementally build the solution, usually in horizontal slices, components or modules.

After each “iteration” we will not have a holistic view of what we’re building.

This is a very common approach by “Agile” teams. In this situation we are deferring the management of risk until we actually have a system that can meet (some of) the needs of the project stakeholders, usually late in the game when the deadline is getting close.

This is not risk management. If we work in this way we cannot work with #NoEstimates.

How do we estimate value without estimating development cost?

OK, so assuming we have the capability and will to deliver vertical slices through a solution early and rapidly, and we have a fixed cross-functional team, 100% committed to the project at hand, we can focus on the potential value of the ideas we want to build while controlling cost using small “drips”.

When we use ROI to decide whether a project is worth pursuing, or which of 2 or more potentially valuable projects we should choose given limited people and resources, we base the “investment” measure on the estimated cost, of which the development costs are part, and the “return” is the value we expect to generate, measured on the same scale as the investment (usually money).

There is a flaw with this approach.

6 months, 2 years, it’s all the same!

Let’s say we estimate a project will take 6 months of development time, costing $500k. We expect that when the product is complete it will generate $2m in revenue. The timing of when that revenue gets generated is key. Will we get anything at all before the product is built in its entirety? Will there be a few months of marketing required after all the features are done before we will start seeing the cash rolling in?

The implication of the timing of value generation is that the actual ROI of what we’re building in a 6-month project might still be negative after 6 months of development time, even if we get everything done that we originally wanted done (and estimated).

Now compare that to, say, a project with an estimated duration of 2 years. After 6 months, the ROI of the two projects will be identical. Our net loss in both cases is $500k, so our ROI is -100%; we have spent half a million bucks with nothing (yet) to show for it.

So, given the erratic, inconsistent and numerous ways we can measure value in software, is the traditional ROI approach an ideal decision making model in this domain?

mortazavi20110206103527187Agile is about early delivery of value, not trying to predict maximum value

The upshot of this is that the less risky approach to generating a positive “ROI” is to work on options that will potentially generate value early, i.e. with relatively small and simple effort. Put simply, if we prioritise initiatives by virtue of which ones we expect to generate value early rather than how much value they will generate over the product’s lifecycle then we do not need to batch these initiatives up into “projects” and estimate how long the project will take.

This can easily be reverse engineered. If our starting point is a “project”, with a list of requirements, the best thing we can do to manage risk (keep our decisions within the bounds of the near, more certain, future) and ensure we deliver value early is to pick the most valuable requirement/problem to solve and come up with a simple, creative approach to fulfilling that requirement in a very short timeframe.

What’s next? One at a time…

The team can go away for, say, 1 month, after which time we holistically assess where we’re at in terms of fulfilling that requirement. What have we learned? Is this requirement still the most valuable one to work on (ignoring sunk costs)? Are we better off ditching what we’ve done and investing in attacking another requirement?

Our measure of what is valuable must reset after each iteration. It’s irrelevant how much we’ve already spent (sunk cost fallacy).

We need to constantly concern ourselves with what is the most valuable thing to do next. This is Agile. This is #NoEstimates.

And this is risk management. Yes, it’s an approach that requires a different way of thinking about how we choose what work to invest in, how much to invest and the decisions we make along the way. But it is risk management nonetheless.

RealEstateBlog360-BigMoneyBut we can’t do this when $200m is at stake!

The #NoEstimates debate has hit a point where the main remaining arguments are around its application in big money projects. Most of the original dissenters – who have now spent time reading more about the ideas put forward by myself and the other #NoEstimates crew – are now in agreement with us that, at least for small scale projects, we can get away with not doing “micro-estimates”, and indeed it may be preferable to work this way.

But when it comes to “macro-estimates” – i.e. how much of the customer’s money are we going to spend – it is argued that a #NoEstimates approach is not viable. That when “you are spending someone else’s money” you need a plan (estimated schedule) to ensure you deliver what is required for the money, with some deterministic level of confidence.

The irony of this argument is that when the big number guys come out swinging with their big numbers, these numbers are estimates! When we call a project that we haven’t yet completed, or even started, a “$200m project”, what we are actually saying is “our customer has a $200m budget and we have to deliver what they want for their money”. In other words, the decision has been made to go ahead, and the budget is $200m. There is no go/no-go decision to be made – it’s already been decided that the project is going ahead, and they want a result for $200m.

For me, with such large sums and timeframes at play, there is all the more reason to manage risk by drip funding small amounts and iterating over a solution in the way I’ve described. Scaling up where required. Tuning and adjusting.

The alternative is to manage risk by using probabilistic estimation techniques based on past projects such as Monte Carlo simulations to derive a total estimated cost with a confidence interval, and then constantly adjust these calculations as the project progresses. But I maintain that the Agile way, where we start from a budget or fixed deadline and then actively build and manage scope along the way, is preferable because it harnesses the creativity of designing and building great software and allows us to welcome and embrace change every step of the way.

Quote-33Create the future rather than predict it

Instead of trying to nail down a plan and predict outcomes, we are forging our own future based on current market conditions at any given time, and the way we feel about what we’ve built so far. We are controlling our costs by working with fixed teams in short timeboxes, and we are constantly assessing the value of what we’re building.

If we work this way we do not need to estimate things up front. Empirical data is being generated as we go along, and we can look at the market with fresh eyes after each iteration. We can see what we’re getting done and what we’re not. We can change our mind on whether we care that we didn’t get the things done that we wanted to get done. We can see which of our assumptions were true and which were false. We can steer our ship in whichever direction we need to avoid the iceberg ahead, while remaining focused on the destination.

This is at the heart of #NoEstimates from my point of view. It is possible to work this way. It is not easy to get to a position where you are able to, but if you can get to that place it is, as Ron Jeffries describes it, “the best known way to work”.

The Ethical Man Month

Systems Thinking tells us that we are products of the system in which we operate. That we will perform based upon the ways we are being measured.

Personally I am astutely aware if the way I am being measured is also a target. I know the measure is not an effective way of helping me contribute to reaching the organisation’s goals.

But the thing I struggle to understand is that if we are gaming the system, and know we are doing so, at what point do our ethics kick in? What is our tipping point?

I once worked with a team that was battling against technical debt. Regression bugs were appearing with increasing frequency due to a lack of automated integration test coverage with legacy systems. My team wanted to do the right thing and fix the bugs that they found, despite the fact that it was not them who created the bugs, but were concerned that they were falling behind with their own work.

They assigned no blame to the unfortunate soul who checked in the code that caused the regression. In fact, they didn’t even get to find out who the culprit was until after time was already spend determining the cause of the bug. There was much complexity in the interactions between components and a gaping lack of integration tests across them. The team just wanted to fix the problem, add some appropriate tests to prevent the problem from happening again, and move on.

The problem for me was that this was impacting on our project schedule. The team were supposed to be working on stories for my project but instead were taking time working on bugs created by other teams. I was being measured on the delivery of the agreed scope in the agreed timeframe, not on our software delivery effectiveness across the portfolio. Surely it was in my best interest to ask the team not to work on other people’s bugs? My delivery schedule was being jeopardised. I would be held accountable for this. I would be asked tough questions. Why didn’t I deliver everything I said I would?

But here’s the thing. Despite how I am measured, I am passionate about creating good outcomes for the stakeholders, the customer and the company, not my specific project. I do not see the work to be done as a set of easily definable story cards. In this and other similar situations I wanted my team, and other teams, to spend time reducing technical debt across the board, improving code quality, collaborating with each other to find ways of making everyone’s lives easier, etc.

I can choose to let the system define me. To be a product of the system. Or I can choose to question things. To think holistically about how we can improve.

The system will reject this. But at least I can go to sleep at night knowing that I am doing what I believe is right.

How much do your ethics influence the decisions you make or don’t make in the workplace?