Why are Software Project Estimates Almost Always Low?

by | September 20, 2019 | Business & Leadership, Project Management

Why are Software Project Estimates Almost Always Low? Here, I’ll explore some theories and truths. The son of a Nobel Prize-winning physicist, Douglas Hofstadter became a prize-winning professor of cognitive science and the author of several books.  In 1979 one of his books described “Hofstadter’s Law.”  Its recursive nature (and its truth) made it popular with software developers.

Hofstadter’s Law

“It always takes longer than you expect, even when you take into account Hofstadter’s Law.”

Any budget manager reading this could be forgiven for thinking, “That’s not Hofstadter’s Law, that’s my law.”  Why is this so consistently true?

The Cone of Uncertainty – in Theory and Reality

In 1958, more than 20 years before Hofstadter published his self-named law, cost engineers proposed a cone of uncertainty to describe estimate accuracy.

The cone of uncertainty illustrates that early in a project the eventual final cost of the project will fall within a range that might be more or less than the original baseline estimate.

As the project plays out, the range of that uncertainty tightens allowing a more accurate estimate.

Project Life

The reality of large, complex software projects is that the original baseline estimate is more likely to be too low than too high.  As the project unfolds, the estimate goes up.

This is what Hofstadter described, but the explanation is more elusive than the well-ordered uncertainty cone.

Three powerful biases help explain why initial estimates are consistently low:

  1. Planning Fallacy,
  2. Confirmation Bias, and
  3. Anchoring.

Let’s look at these biases and some remedies for their impact.  You can Google any of the items below for more information, or use “Google Scholar” if you are interested in the studies behind the summaries.

Biases

Planning Fallacy

When it comes to planning, people are optimists and optimism is a bias.  “Planning fallacy” describes the tendency of plans to understate time and cost while overstating benefits because of this optimism.

Optimistic plans assume that nothing will go wrong, an assumption that almost never matches reality.

Some characteristics of this phenomenon are important in counteracting it.

  • The optimism bias does not extend to the efforts of other people (“My work will go smoothly, I’m not so sure about yours”).
  • People will give more weight to their (optimistic) plans than to their own experience.

Confirmation Bias

An estimate may involve many factors.  When there is strong interest in a lower estimate than a higher one, the desire to produce a lower estimate can cause people to discount factors that would drive the estimate up and give more weight to factors that support a lower estimate.  This is confirmation bias, also described as motivated reasoning.

Drivers to produce a low estimate are common:

  • There is a need to get the new software in place quickly.
  • The cost of a project is a heavily weighted factor in a competitive bidding process.
  • Project approval depends on low cost.

Anchoring

A project manager asks a developer for an estimate.  “I’m thinking it’s about 40 hours of work, but your opinion is the one that matters.”  It was already too late to get a completely unbiased estimate (because there is no such thing, and regardless of the stated good intentions), the project manager has added anchoring bias to the list of biases already affecting the estimate.

Anchoring bias is the tendency that people have to give more weight to the first piece of information they get than information they get later, and this bias affects people whether the first piece of information they get is well-reasoned or not.  Once the project manager tells the developer that they think the work should take about 40 hours, the developer may adjust it up or down from the 40 hours, but 40 hours will be the ‘anchor’.

Counteracting Bias

Awareness

Being aware of biases is a significant step towards counter-acting them.  Awareness moves an unconscious bias to the conscious mind.  You can ask yourself if your motivations are driving your optimism, or if you have adequately considered what might go wrong.

Use of Historical Information

Find similar completed efforts from the past and use them as a basis for the estimate.  The time it took to complete something in the past is an excellent indicator of how long it might take to complete a similar task in the future.  Be careful of discounting away any impediments the previous effort encountered; better to use any hurdles the previous effort as examples of things that might challenge the new work.

The Devil’s Advocate

“Playing the Devil’s advocate” is an expression that has its origins in a practice that goes back over 400 years.  In the Catholic Church, “the Devil’s advocate” had the role of finding character flaws in candidates nominated for sainthood.

In estimating a project or a task, you can assign someone the role of “Devil’s Advocate” and ask them to look for things that could go wrong.  This should be someone who has strong interest in a low estimate.

If possible, the Devil’s advocate should come up with their own independent estimate.

Multiple, Independent Estimates

If possible, get estimates from alternate sources.  Ideally, these sources should be as free from bias as possible.  They should (a) not know the other estimates, and (b) not have a strong motivation to produce a low result.

Conclusion

Avoiding estimate bias can be a significant amount of work.  In addition to the extra resources that might be involved in developing an estimate, those people will need to have enough background to weigh in knowledgeably. It is worth it.  An accurate estimate can make the difference between a fun project and a stressful one, between success and failure. If you have more questions about bias, reach out to us at Stoneridge Software.

*Full diagram

 

Related Posts

0 Comments

Submit a Comment

Your email address will not be published. Required fields are marked *

Upcoming Events

september

02sep10:00 am10:30 amThe Modern Manufacturer - Cycle Count Management

02sep12:00 pm12:30 pmBest Practices for Work Order Planning and Routing for Field Service Companies

09sep10:00 am11:00 amWhat is Levridge? An Overview of the Ultimate Ag Solution

09sep12:00 pm12:30 pmOverview of Microsoft Promotion for NAV and GP Clients

10sep11:00 am12:00 pmConfab with Stoneridge - Livestream - Field Service Comparisons

16sep10:00 am10:30 amThe Modern Manufacturer - Product Lifecycle Management

16sep12:00 pm12:30 pmHow to Upgrade to Dynamics 365 Finance and Operations: Examining Custom Code and ISV Implications

16sep2:00 pm2:30 pmScheduling and Dispatching for the Modern Field Service Company

23sep12:00 pm1:00 pmDynamics 365 Business Central Wave 2 Release Preview - An Overview of the Most Exciting New Features Coming in October

24sep11:00 am12:00 pmConfab with Stoneridge - Livestream - The Modern Workplace: Teams, Sharepoint, Outlook, Office, and AI Integration

30sep10:00 am10:30 amThe Modern Manufacturer - Return Management

30sep12:00 pm12:30 pmEmpowering Your Field Service Technicians with a Mobile Solution

october

07oct12:00 pm1:00 pmThe Three Paths to Business Central from Dynamics GP

08oct11:00 am12:00 pmConfab with Stoneridge - Livestream - The Vision and Strategy of Microsoft Business Systems

14oct10:00 am10:30 amThe Modern Manufacturer - Complex Cost Modeling

14oct12:00 pm12:30 pmGenerating Custom Inspection or Process Forms

19octAll Day22Stoneridge Connect Fall 2020

22oct11:00 am12:00 pmConfab with Stoneridge - Livestream - Stoneridge Connect Recap

28oct10:00 am10:30 amThe Modern Manufacturer - Engineering Change Orders

About Stoneridge
Stoneridge Software is a unique Microsoft Gold Partner, with emphasis on partner. With specialties in Microsoft Dynamics 365, Microsoft Dynamics AX, Microsoft Dynamics NAV, Microsoft Dynamics GP and Microsoft Dynamics CRM, we focus on attracting the most knowledgeable experts in the field to our team, and prioritize delivering stellar solutions with maximum impact for your business. At Stoneridge, we are deeply committed to your results. Each engagement is met with a dedicated team, ready to provide thorough, tailored, and expert service. Based in Minnesota, we intentionally “step into your shoes,” wherever you are. We focus on what you care about, and develop trusting, long-term relationships with our clients.

Subscribe To Our Blog

Sign up to get periodic updates on the latest posts.

Thank you for subscribing!

X