Well it’s a question decided since 1994 by Chaos Reports from the Standish Group.
Jim Johnson (founder of the Standish Group) undertook research on IT project failure. This was in 1994. Jim and his team wanted to find out what was the cause of project failure and find ways to reduce it. By 2006 they had amassed a huge amount of data.
In their 1994 report, they found that 84% of projects failed or had challenges. 45% of developed features were never used. 31.1% of projects faced cancellation before they ever got completed. 52% of projects cost 189% of their original estimates.
Since then their statistics on project failure settled around the 70% failure mark. The main culprit identified was poor planning or failing to follow good planning.
Over time their definition received criticism for focusing on time, scope and budget. This was a very traditional model that you see in tools such as Microsoft Project.
They continued to review their methods and stopped publishing the reports in 2015.
They needed to make a change.
Why make a change?
The very first Chaos reports painted a pretty depressing picture of IT projects. Also they didn’t tell you what to do about it. YouTube is full of videos about Mega-Project failures. They’re kind of fascinating to watch. Especially from a project management perspective. They don’t tell you either.
Although failure is possible, it is also largely avoidable with a bit of planning. The Chaos reports did say that. Though they also pointed to a ‘crisis’ that critics suggested to be more illusion than reality.
This crisis model had a concept. The concept that all projects exist as successful, challenged or failed. They based all findings on time, scope and budget constraints. This was considered too narrow by critics. The metrics seemed to be symptoms, rather than the cause.
Software development has changed in 30 years. That’s not even considering the effect that AI is having. It’s not guys in white lab coats standing around a tape to tape cabinet, like a casting call for “Lost in Space”.
This method doesn’t account for scope changes, technology advances or shifting needs.
The 2015 report concludes:
Over the last 20 years the project management field has experienced increasing layers of project management processes, tools, governance, compliance, and oversight. Yet these activities and products have done nothing to improve project success.
Modern projects have a different perception. They can prioritise meeting stakeholder needs. They can focus on delivering high-quality, ROI driven projects. Those that maintain a healthy work environment.
A large part of this is the rise of Agile. It’s obvious in new Standish Group reports that they’re fans.
The Agile method takes an iterative approach to delivering IT projects. It emphasises collaboration, flexibility and a customer-centric approach. It can handle evolving circumstances.
This approach requires even better documentation. You need to onboard changes against the goals of the project. You still need the original requirements documentation.
What’s a requirements doc?
A Requirements Definition Document (RDD) specifies the Business, Functional and Technical requirements. What the goals are, what it actually has to do and the technology and conditions that have to exist to make it work.
In the Functional section, who the System Actors (people and things) are. How they’re going to interact with the system and the user experience that supports it.
Project Success Reborn?
After shutting down the Standish Group came back in 2021 with a new report. The title: “CHAOS Report Beyond Infinity”. If you’ve got USD $697 to spare: Click Here to see.
This 260-page printed book is in full colour with hundreds of charts, tables, and graphics. We start with the three new Factors of Success, Good Place, Good Team, and Good Sponsor. Then we look at the root cause of project failure which is poor decision latency. Then we look at the root cause of poor decision latency which is low emotional maturity
OK that’s a pretty big pivot. The Standish Group have aligned themselves with the principles of Agile. There is an emphasis on delivering customer value. The name of this approach: “Flow Centric Project Management.”
Flow Centric Project Management
The ‘flow’ method focuses on continuous change in small increments. This reduces the role of traditional budgets, plans and managers. The goal is to reduce project overhead by delivering continuously.
Such an approach gains better alignment with product management. These concepts exist there as well.
The sub-projects in the flow become “Epics”.
In Agile “Epics” are large works that share a common goal. They break down into smaller tasks called “user stories.” They can span several sprints. A sprint is a short time to complete a set of work (less than a month, usually 2 weeks). The sprint lead is a “Scrum Master”. That’s a person who runs the sprint and the ‘stand up’ meetings where developers get together to discuss the work. At the end of the sprint they hold 2 meetings. One to demonstrate the work for the stakeholders, and another for an internal review.
Product, Development and Customer/User Experience are now connected. The Epics help align the work with the business goals. Having the common language helps the teams keep focus on the business goals.
How do you look at it now?
Other new voices have had their own research. A new report has emerged called, “The impact of size and volatility on IT project performance”. It comes from Chris Sauer, Andrew Gemino, and Blaize Horner Reich. Their research took the approach that only one third of IT projects ‘fail’.
The data came from project managers rather than executives. Project managers should have more knowledge about a project than executives. That’s the theory. The research only covered recent projects for the freshest knowledge and recollections.
The findings related to:
Project Size
They measured the effort, duration, team size and budget. They found that the larger a project is, the greater the risk. They found this no matter the experience of the project manager. They found a focus on team size, effort and duration limited the risk. They found that 25% of projects still underperformed no matter the size.
Volatility
They looked at governance (measured in changes by project manager or sponsor). They looked at target (measured by changes in schedule, budget and scope).
- Projects with no changes in personnel still faced a 22% risk of underperforming. This rose to 50% when two or more personnel left.
- Projects with 9 or fewer changes of any type faced a 33% risk of underperforming. Projects with over 9 changes of any type also rose to a chance of 50% failure.
Volatility is strongly related to the success of IT projects. Project governance and the documents that support it are still important.
IT Project Managers are not the whole story. Management and steering committees are important to risk management. Managing ambition, moving targets and management turnover present a challenge for IT projects.
Agile is effective when someone says “I need this”. By the next sprint they might say, “maybe I didn’t need this so much, or something else is better.” It’s quicker to the pivot. This is called variable scope.
It looks good because things appear to be getting done. Although having some goals in the first place seems to be key. The balance allows for innovation and ‘trying things’. The governance keeps the project on track. It’s a company culture thing.
OK sounds great, now what?
There’s no doubt that everyone is a fan of Agile. Though there’s a good chance that not everyone understands a nuanced view of it. It’s also vulnerable to ‘creeping bureaucracy’. Agile is not a licence to throw governance out the window.
It does allow for innovation, research and more conversations around success. It’s important to have methods that deliver success in a modern setting.
It’s not the beginning of a new era. It’s an evolution.
Still. “Don’t screw it up alright!”
Virgil Reality…