Why are Software Project Estimates Almost Always Low?

by | September 20, 2019 | Business & Leadership, Project Management

Why are Software Project Estimates Almost Always Low? Here, I’ll explore some theories and truths. The son of a Nobel Prize-winning physicist, Douglas Hofstadter became a prize-winning professor of cognitive science and the author of several books.  In 1979 one of his books described “Hofstadter’s Law.”  Its recursive nature (and its truth) made it popular with software developers.

Hofstadter’s Law

“It always takes longer than you expect, even when you take into account Hofstadter’s Law.”

Any budget manager reading this could be forgiven for thinking, “That’s not Hofstadter’s Law, that’s my law.”  Why is this so consistently true?

The Cone of Uncertainty – in Theory and Reality

In 1958, more than 20 years before Hofstadter published his self-named law, cost engineers proposed a cone of uncertainty to describe estimate accuracy.

The cone of uncertainty illustrates that early in a project the eventual final cost of the project will fall within a range that might be more or less than the original baseline estimate.

As the project plays out, the range of that uncertainty tightens allowing a more accurate estimate.

Project Life

The reality of large, complex software projects is that the original baseline estimate is more likely to be too low than too high.  As the project unfolds, the estimate goes up.

This is what Hofstadter described, but the explanation is more elusive than the well-ordered uncertainty cone.

Three powerful biases help explain why initial estimates are consistently low:

  1. Planning Fallacy,
  2. Confirmation Bias, and
  3. Anchoring.

Let’s look at these biases and some remedies for their impact.  You can Google any of the items below for more information, or use “Google Scholar” if you are interested in the studies behind the summaries.

Biases

Planning Fallacy

When it comes to planning, people are optimists and optimism is a bias.  “Planning fallacy” describes the tendency of plans to understate time and cost while overstating benefits because of this optimism.

Optimistic plans assume that nothing will go wrong, an assumption that almost never matches reality.

Some characteristics of this phenomenon are important in counteracting it.

  • The optimism bias does not extend to the efforts of other people (“My work will go smoothly, I’m not so sure about yours”).
  • People will give more weight to their (optimistic) plans than to their own experience.

Confirmation Bias

An estimate may involve many factors.  When there is strong interest in a lower estimate than a higher one, the desire to produce a lower estimate can cause people to discount factors that would drive the estimate up and give more weight to factors that support a lower estimate.  This is confirmation bias, also described as motivated reasoning.

Drivers to produce a low estimate are common:

  • There is a need to get the new software in place quickly.
  • The cost of a project is a heavily weighted factor in a competitive bidding process.
  • Project approval depends on low cost.

Anchoring

A project manager asks a developer for an estimate.  “I’m thinking it’s about 40 hours of work, but your opinion is the one that matters.”  It was already too late to get a completely unbiased estimate (because there is no such thing, and regardless of the stated good intentions), the project manager has added anchoring bias to the list of biases already affecting the estimate.

Anchoring bias is the tendency that people have to give more weight to the first piece of information they get than information they get later, and this bias affects people whether the first piece of information they get is well-reasoned or not.  Once the project manager tells the developer that they think the work should take about 40 hours, the developer may adjust it up or down from the 40 hours, but 40 hours will be the ‘anchor’.

Counteracting Bias

Awareness

Being aware of biases is a significant step towards counter-acting them.  Awareness moves an unconscious bias to the conscious mind.  You can ask yourself if your motivations are driving your optimism, or if you have adequately considered what might go wrong.

Use of Historical Information

Find similar completed efforts from the past and use them as a basis for the estimate.  The time it took to complete something in the past is an excellent indicator of how long it might take to complete a similar task in the future.  Be careful of discounting away any impediments the previous effort encountered; better to use any hurdles the previous effort as examples of things that might challenge the new work.

The Devil’s Advocate

“Playing the Devil’s advocate” is an expression that has its origins in a practice that goes back over 400 years.  In the Catholic Church, “the Devil’s advocate” had the role of finding character flaws in candidates nominated for sainthood.

In estimating a project or a task, you can assign someone the role of “Devil’s Advocate” and ask them to look for things that could go wrong.  This should be someone who has strong interest in a low estimate.

If possible, the Devil’s advocate should come up with their own independent estimate.

Multiple, Independent Estimates

If possible, get estimates from alternate sources.  Ideally, these sources should be as free from bias as possible.  They should (a) not know the other estimates, and (b) not have a strong motivation to produce a low result.

Conclusion

Avoiding estimate bias can be a significant amount of work.  In addition to the extra resources that might be involved in developing an estimate, those people will need to have enough background to weigh in knowledgeably. It is worth it.  An accurate estimate can make the difference between a fun project and a stressful one, between success and failure. If you have more questions about bias, reach out to us at Stoneridge Software.

*Full diagram

 

Related Posts

0 Comments

Submit a Comment

Your email address will not be published. Required fields are marked *

Upcoming Events

november

05nov11:00 am12:00 pmConfab with Stoneridge - Livestream - Project Oakdale: The NEW Integration of Power Platform into Teams

18nov10:00 am10:30 amThe Modern Manufacturer - Tears and Trauma of MRP

19nov11:00 am12:00 pmConfab with Stoneridge - Livestream - Project Oaktree: NEW Functionality for Dynamics 365 Manufacturing

december

02dec9:00 am12:00 pmOnline Workshop - Dynamics GP Year End Close (Morning Session)

03dec11:00 am12:00 pmConfab with Stoneridge - Livestream: Live Agents, Power Virtual Agents, Omnichannel – Oh My!

03dec1:00 pm4:00 pmOnline Workshop - Dynamics GP Year End Close (Afternoon Session)

09dec10:00 am10:30 amThe Modern Manufacturer - Manufacturing Policy Management

17dec11:00 am12:00 pmConfab with Stoneridge - Livestream - Technology in 2020: A Year in Review

About Stoneridge
Stoneridge Software is a unique Microsoft Gold Partner, with emphasis on partner. With specialties in Microsoft Dynamics 365, Microsoft Dynamics AX, Microsoft Dynamics NAV, Microsoft Dynamics GP and Microsoft Dynamics CRM, we focus on attracting the most knowledgeable experts in the field to our team, and prioritize delivering stellar solutions with maximum impact for your business. At Stoneridge, we are deeply committed to your results. Each engagement is met with a dedicated team, ready to provide thorough, tailored, and expert service. Based in Minnesota, we intentionally “step into your shoes,” wherever you are. We focus on what you care about, and develop trusting, long-term relationships with our clients.

Subscribe To Our Blog

Sign up to get periodic updates on the latest posts.

Thank you for subscribing!

X