Estimates are still a painful subject in the world of software development. Some people hold on to techniques from the past, trying to predict the timeline of a software project. Others are in favour of changing these habits and are looking for better ways to become predictable.
In this article, I’ll give you my perspective on this delicate subject.
We are asked when some piece of work will be finished. Finished? What do you mean with finished? You mean when all solutions to known requirements are built and deployed? Or do you mean when the customer achieves his goals?
There is one answer to that question. But the answer is twofold. A software product is never finished, but it’s good enough when the customer achieves their goal(s) and tells us it’s enough.
And achieving goals is done by building great software, guided by feedback from your customer.
If only we had a crystal ball and could look into the future, then we could answer the question one hundred percent accurately.
In the ‘waterfall’ era, time and scope were considered fixed. A project had to be ready on time and ‘resources’ were allocated to perform all known and defined tasks. Together with budget, this is called the Triple Constraint. I have never found this a satisfying way of working with these constraints and thankfully it has changed. When working in an Agile setting, the scope can be considered flexible, time and budget being fixed to some extent. In this way you can make an agreement with the customer that the features with the highest priority are built first. During this period, the customer is frequently providing feedback and the direction of the product can pivot if necessary. This is the whole idea behind Agile in the first place. When keeping the scope variable, together with the customer you can create a product that contains the most valuable features within an agreed timeline, so the budget isn’t exceeded. The only challenge remaining, is how to predict the contents and the volume of that flexible scope.
Triple constraint; Agile vs Classic
Scope is the accumulation of all implemented requirements. In the modern Agile world, we have an idea what that accumulation will look like in the end. But it’s just an idea and together with the customer we agree on uncertainties regarding the requirements, thus scope. In iterations we can try to predict what the team can deliver at the end of each one. The smaller the requirements and iterations are, the better we can predict its outcome. These short iterations, small user stories and frequent feedback from the customer, are in fact a way to try to mitigate the uncertainty.
| Years ago, a Project Manager asked us to estimate all the tasks in a two-year project, so he could finish up his Ghantt chart. We were rookies so we all gave estimates. Estimates based on nothing. Guesstimates. The project kicked off and immediately, we got into trouble because of normal software development obstacles. The Project Manager started pushing the team, because his chart didn’t represent the reality anymore. And his job was on the line, higher management wanted him to successfully finish this project within the timeline given. More issues raised, so more pressure from the managers. This ended up in a 'blame game' with a horrible outcome, because as we suffered from his dominance, the quality of the product and thus the customers did too. One of the main reasons these kinds of project fail, is the lack of understanding about estimates and fixed scope. This results in no room for failure and learning and to mitigate possible obstacles in software development. People started to protect themselves by putting slack into their estimates. Projects became largely overestimated and customers backed off or became unhappy. |
Okay, this happened years ago; the world has changed with the introduction of Scrum, but I’m still encountering the same estimation habit.
Cone of Uncertainty
A widely used illustration to show the uncertainty within software development is the cone of uncertainty. This cone represents the variability in estimates. At first, the uncertainty is at its highest and this will reflect in the variation in estimates. When time goes by, things become clearer and thus the uncertainty will decrease as well.
Cone of Uncertainty
This nice graph demonstrates why we should forget about the idea that we can accurately predict.
Commitment
A customer would love to have certainty about the product, his budget and timeline. How do we give the customer that feeling?
One of the Scrum values is commitment.
Commitment from the Product Owner. Commitment from the team members. These team members are committed to create the best possible product, given all circumstances. This commitment is actually your guarantee to the production of a good product. And when there is a level of trust, trust that goes both ways, the project will have its essential ingredients. The missing reliability in the estimates is replaced by trust in the relationship between the parties working on the product.
Productivity
Another reason for which I see people using estimations, is trying to measure productivity.
But what is productivity anyway? The number of points estimated versus points deployed? The number of stories done? Your velocity? Your velocity doesn’t say anything about the business value you’re aiming for. In fact, there is no link at all. You can have a very high velocity, but at the same time not delivering any value. Estimating upfront and comparing with the time spent on a requirement, has nothing to do with productivity.
I’m trying to avoid the term productivity at all. Because our job is to create the best possible solution with the best possible results for our customers. So, there is no need to measure or compare anything, except your goals. If you want to have this productivity indicator, please consider it as bringing value, reaching your goals, increasing the customer satisfaction, etc.
Past projects as a reference
“You can give me a good estimate, because you built this before.”
Did you actually build this before? Yes? Are you sure? Is everything the same as it was back then? Same requirement, same frameworks, same team, same PO, same customer, same goals, etc.? Yes, okay, then don’t build it again, but use that solution again. Problem solved.
If it’s not the same, then you could have some similarities with work you did in the past. Which is great, you can use that as a starting point in your new work and you can share your experiences with the team and make sure you learn from the previous time. Developing something for the first time, takes time and correcting mistakes also takes time and effort, causing delayed delivery to the customer.
Misuse
Estimations can be misused and be interpreted as deadlines or promises when you publish them. This will result in more pressure for the teams, which causes a negative effect on the product and in the end on the customer. When you give estimates, make sure this information is used correctly. If you’re not sure what happens with it, ask. Question the need for estimates, how it contributes to the goals and provide alternatives.
Planning poker
Planning poker is a step forward. Instead of exact hours, you now estimate in complexity and use a range that looks like the Fibonacci numbers. The reason for using the Fibonacci sequence is to reflect the inherent uncertainty in estimating larger items. In planning poker there are a few other rules, like simultaneously showing your poker cards and repeating the estimation process until a consensus is reached.
I mentioned that it is a step forward, but I’m still not a big fan of the planning poker mechanism. It’s still very unreliable and could facilitate intimidation by more experienced team members, including the Product Owner. And there is a big chance that team members wait on their peers to show the card… I’ve seen teams doing planning poker but still adding up the different expertise within one team.
There is one good reason to do planning poker with the team. And that is the discussion after showing your card and noticing there is a big different in size of estimates. By discussing the requirement, you make it clearer for everyone. Questions arise and can be answered. Preliminary solutions can be evaluated and so on.
So why not call it Refinement poker. I’m not kidding, really, let’s do that.
Manipulation
Don’t forget that estimations can be manipulated. When there is a blame culture or the team is being micromanaged and (the wrong) productivity is monitored constantly, there is a very big chance the estimates will be manipulated. You want a higher velocity? Sure, you will get a higher velocity. No doubt. It’s these old patterns that mainly causes the manipulations. I really can’t blame the developers for doing this.
Alternatives
There are other ways to aim for predictability.
Story count
When your product backlog items (stories) are on average of the same size, you could consider counting the stories for your sprints. I’m a big fan of this alternative and I frequently use it.
Make sure the stories you’re writing are well refined, correctly sliced and small enough to comprehend. The smaller they get, the better they become an average size story.
If the customer wants us to use estimations, story counting is for me the best way to plan sprints, epics and even projects.
T-shirt sizing
T-shirt sizing is a pretty powerful way to quickly size epics. The sizes can then be used while creating a roadmap for your product.
Giving a T-shirt size to epics or stories motivates the PO and team to slice the stories in smaller pieces, so it becomes more clearer and tangible.
| The smaller your t-shirts are, the more you can pack in a suitcase. |
NoEstimates
#NoEstimates is a movement that stands for changing the habit of old-fashioned estimates.
The name is a bit misleading and suggests that you don’t estimate at all.
What they are trying to accomplish is to let people think about the concept of estimates and come up with better ways to become predictable, in a way it makes sense.
Kanban
In my article about Kanban, I’ve explained what Kanban is all about. In Kanban you don’t need estimations. A combination of WIP (work in progress), JIT (just in time) and flow, creates an indicator which can be considered as your cycle-time. With this substantiated cycle-time you’ll be able to do forecasting. Just like in Scrum, a team using Kanban is constantly looking for improvements of the process and aiming for a better flow and constant pace, you can keep up indefinitely. This flow and ability to change makes sure you’re being reliable as a team.
Estimates in scaling
As requirements are vague, your estimations will be as well. Next to that, it’s time-consuming and unnecessary in most cases. In a scaled environment, which is very complex on its own, there are many kinds of dependencies between requirements and teams, which makes forecasting even more important, but a lot more difficult. But for example, why does SAFe tells us to estimate the product backlog items with all the known consequences? If you scale your Agile environment, you scale the uncertainty with it. Don’t let estimations become your pitfall, try to think of a better way to forecast. Story count could very well work in scaled agile settings.
Can we do without estimates?
In my ideal world we can do without estimates.
We develop a product incrementally, feedback-driven and prioritised, based on business value.
That’s it basically.
All the time spend in estimating stories or other types of requirements, or having estimates, doesn’t make your product a better one. It only slows down the process.
If the customer trusts the team, the team is professional, mature, committed and supported by the organisation, the product can be released a.s.a.p.
Not convinced?
If you’re still not convinced and want to keep estimating the old way, or you’re even forced to use this time-consuming and unreliable manner of estimating, let me give you a few tips. Use the estimations wisely for team purposes like evoking refinement discussions. Don’t let your estimations become a KPI for management. And question the need for these estimates. Experiment with alternatives and discuss the everlasting presence of uncertainty.
