One of the biggest concerns companies face are overblown IT budgets and missed deadlines. Regardless of its size and magnitude, every software project faces the risks of failing to meet requirements and waste invested time and effort into a product that brings zero to no value.
Psychologists have long studied the human incapacity to plan adequately. The phenomenon is called the "planning fallacy."
Wouldn't it be nice to plan better? Wouldn’t it be nice to diagnose the planning fallacy in the early stages?
This article aims to bring out Mad Devs' expertise and explain why projects don't meet deadlines. We’ve learned them the hard way and have long changed our approach to managing and developing IT projects. Although Mad Devs has a long journey ahead of us, we’ve started documenting our knowledge to share with companies that face similar challenges.
Without further ado, here are three reasons why IT projects are late and exceed initial budgets.
Uncertain project scope
Unclear product requirements and project scope are the most common causes of overblown software budgets. Without a clear product vision, the uncertainty affects the budget estimates as the work scope is continuously re-estimated. Note that such changes can occur due to both internal and external reasons.
Here we listed the most frequent reasons that drive uncertainties:
Market forces: Especially startups without market fit are keen on adapting their product and continuously changing vision, monetization model, target audiences, and even investors. Thus, such companies redesign and redevelop the same product to find the right market fit.
Miscommunication with the development team: The delivered product, module, or feature doesn’t meet customers’ expectations. Although agile methodology with its short feedback loop helps both parties to avoid such problems on earlier stages, an inexperienced team without established communication processes often can fail to identify and understand customers’ needs and end up having to re-develop the same feature over and over again.
Negative feedback from end-users: Even meticulous market analysis doesn’t guarantee that existing users will understand the software’s value. Taking negative feedback and truly building a user-oriented product is impossible without re-arrangements in a product roadmap, which ultimately extends the time estimates and budget.
Obsolete organizational and development processes
Everybody knows the story of how the FBI's Virtual Case File wasted 170 million of taxpayers’ money in 3 years. Let’s give a bit more context of what happened next. A new team implemented SCRUM and successfully finished the project in 1 year with 12% of the original budget.
Organizational and development processes are key factors that boost the development speed and thus reduce costs. Note that agile methodologies are not a panacea for all that is wrong with software development. But there are legitimate reasons why agile, when applied properly, brings benefits such as efficiency and transparency of software development, delivery, and support.
How to understand if your software team fails to follow best practices in communication and development? Here is a generic list of bad practices that accumulate troubles:
- Tasks are distributed and communicated in private chats or orally without leaving any traces on paper.
- No CI/CD. Thus app builds are sent manually upon a manager’s request instead of automatically via pipeline. As a result, some features don’t make it to the release.
- No documentation. IT teams send oral instructions and explanations to each other without effective knowledge management.
- Lack of observability. Thus, the team learns about production failures from users’ complaints because the system has no configured monitoring.
- Long feedback loop. IT teams wait a long time to develop a feature only to discover that they didn’t understand the feature specifications.
- Poor tasks estimations. Thus the team can’t provide reliable deadlines.
Lack of proper infrastructure cost planning and management
“Any system has a risk of failure. The bigger the system, the more tangible the risk, as the growing number of operations means a growing number of things to go wrong.”
Customer base and load on infrastructure are positively related. Business growth without proper management of monthly expenses on SaaS, PaaS, and IaaS services leads to unoptimized resource consumption.
Optimizing infrastructure costs can be a tricky task. DevOps engineers are focusing on either improving performance or cutting costs. It’s flipping all the time, and you need to find a balance between effectiveness and cost benefits.
- Improper networks and connectivity
- Lack of efficient data storage architectures
- Compute management and provisioning
- Data acquisition problems
- Lack of some powerful computing platforms
At the same time, without proper infrastructure management, the following issues may occur:
From our experience of working with large-scale IT projects, the above mismanagements occur in monthly expenses ranging from $500-$20,000 depending on the severity of the situation and project size.
For example, companies in early stages without IaC practices create an environment where some developers can quickly launch VMs or services and later forget about them until the bill arrives. In contrast, companies in mature stages face expenses that don’t root in only one problem and require a long-term approach to optimizing infrastructure. The most prominent example is one of our clients — Veeqo, an inventory and shipping platform for e-commerce. You can read a comprehensive case study here.
To wrap up
95% of problems can be fixed with proper communication and development processes. Since 2004, we’ve worked on dozens of highly-loaded IT projects and have come up with practical ways to mitigate risks, adequately plan project scopes, and build a corporate culture of rock stars. Although Mad Devs has a long journey ahead of us, we’ve started documenting our knowledge to share with companies that face similar challenges.
In this article, we’ve summarized a very generic list of common reasons why your software projects are late and exceed initial budgets. We hope you’ve found them helpful.