Why you should be afraid of software project failure

By Peter Meyer, Principal Consultant, Elkera Pty Limited

Rates of software project failure are alarming

It is widely accepted that 50% or more of software projects fall short of their expectations to a serious degree and that many projects fail completely. This claim is not just anecdotal. Confirmatory evidence gathered by the Standish Group is summarised in an article by Jorge Domingues at ProjectSmart UK: The Curious Case of the CHAOS Report 2009.

According to The Standish Group CHAOS Reports cited in that article, only around 30% of software projects are rated as successful. Most projects fall short in some way. Depending on the tests applied, around 20% – 25% of projects fail completely.

Even if the success rate is somewhat higher, those figures are shocking.

Any business manager should be alarmed by those figures.

The consequences of software project failure can be severe. Large sums are invested in projects that deliver little value or are scrapped. Personnel devote huge efforts to projects that produce little value.

The fact that many software projects fail is well known and widely discussed. In that context why, after decades of work since the introduction of computer technology, does the problem continue?

There is something seriously wrong in the software world. Something has to change in the way many businesses approach software projects.

The focus is on small and medium enterprises

The discussion in this and the following topics is based on these assumptions:

The project is to procure software from an external supplier or developer.
The procuring organisation does not have its own expert business analysts and project managers.
The procuring organisation does not develop software in-house.

In-house development by experienced teams who understand the business is completely different to the situation faced by an enterprise seeking to procure software from an external solution provider.

Externally sourced software projects have a life cycle that spans business process analysis, software requirements elicitation, procurement, development, deployment and maintenance. Failure can occur at most of those stages.

There is one further assumption:

Only those risks that arise in the business process analysis and requirements elicitation stages of software project, up to the point when the organisation is ready to begin procurement, are considered.

Software project failure risks for SME software projects

Some common sources of software project failure are listed in an article in Forbes Magazine, “14 Common Reasons Software Projects Fail”: .

That list is a pretty good starting point, but it is far from complete.

It may be helpful to consider four main categories of risks that affect software project planning:

Project management risks.
Business change planning risks.
Business context risks.
Requirements elicitation and documentation risks.

We will focus just on the core or most significant risks in each category.

Project Management Risks

Core project management risks include:

Using unskilled personnel for planning.
Setting unrealistic deadlines, which leads to incomplete analysis or delay.

Business Change Planning Risks

Business change planning risks include:

Failing to understand and clearly define the business problems that are to be solved.
Failing to set clear, measurable objectives (success factors) for the project.
Failing to align the objectives of a discrete project with organisation wide needs (failing to have or align with a master plan)
Failing to limit project scope to the capacities of the enterprise.
Failing to establish a clear business case with a positive return on expected investment.
Failing to analyse and plan to optimise business processes affected by the proposed software.
Making unfounded assumptions about the technology or product required for the solution, including expecting to use emerging technologies.
Failing to include important business process stakeholders in planning.
Failing to focus on customer or end user needs.

Software projects require change. If the need for change is ignored or underestimated, all subsequent planning work is undermined.

Business Context Risks

Business context risks include:

The solution would require bespoke software.
The solution would require heavily customised, off-the-shelf software.
The business operates in a rapidly changing business environment that either prevents a clear definition of requirements up front or may lead to significant change during the project.

Business context risks increase project complexity and the risk of failure.

Requirement elicitation and documentation risks

Finally, we come to the requirements elicitation and documentation risks. They include:

Failing to prioritise requirements based on business value.
Including user ‘wish list’ requirements without critical assessment of need.
Failing to include the interests of all stakeholders.
Failing to include business context information with requirements so that solution providers understand the “why”.
Failing to include all requirements, including non-functional requirements, data migration requirements and other change requirements.
Basing requirements on unjustified technology assumptions.
Failing to define requirements in a form that allows stakeholders to reliably assess if the requirements are satisfied by a delivered solution (ambiguous requirements).
Failing to align requirements with the nature of the expected solution and applicable assumptions. The requirements for an off-the-shelf system will be quite different to those for a bespoke or heavily customised system.

Defective requirements will certainly undermine the value of the delivered solution. If severe enough, they may completely undermine the project to the point of failure.

Looking at the four risk categories and their specific risks, is it any wonder that so many projects under-perform or fail? Planning for new software is a very complex process.

The biggest source of software project failure

Using unskilled personnel for such a complex process is likely to be the biggest single source of software project failure for SMEs because it magnifies every other risk. It is not realistic to expect unskilled personnel to navigate each of those risks to achieve a successful outcome.

The specific risk of using unskilled personnel for planning is considered in the next article: Unskilled personnel; the biggest risk of all to your software project.