Hybrid Project Management: A Middle Ground Between Agile and Waterfall

Is the Cold War between Waterfall and Agile thawing? At its heart, the difference between the two project management methodologies is predictability versus adaptability. Waterfall strives for predictability: Consider the project finished only when all predetermined features are completed and fully implemented. Agile strives for adaptability: Deliver a minimum viable product (MVP) and release new features in an iterative fashion to collect the user feedback that will guide the path of improvements.

What Is Agile-Waterfall Hybrid?

Since being introduced officially more than 20 years ago, Agile has continued to grow in popularity. Widespread throughout software development, the practice has been creeping into areas where Waterfall methodologies have remained dominant. A hybrid approach formalizes the experimentation of using Agile in a larger Waterfall structure.

“True hybrid is a combination of predictive and uncertain work,” says Jim Stewart, a Boston-area Toptal project manager, Scrum master, Agile coach, and instructor. In a hybrid model, Waterfall techniques are used for the well-understood, predictive parts of the project whereas Agile techniques are used for the iterative, more uncertain ones.

In this way, a company can create software in an Agile fashion but roll it out in a Waterfall process. A financial company might develop a product that has several large components that need to be audited and thus cannot be released until they are completed and approved. At the same time, smaller pieces of that larger feature, as well as other aspects of the user interface, can be iterated by the development team through a series of sprints.

When it comes to setting up the right hybrid system for your project, flexibility is key. “I don’t believe in something one-size-fits-all,” says Miroslav Anicin, a Belgrade, Serbia-based project manager who is a part of Toptal’s Project Management screening team and a contributor to this blog. “You can’t go somewhere and apply methodologies by the book. You need to know exactly how to tailor those approaches based on different factors such as team maturity, company culture, subject cultures, the type of project, the size of the team, and the size of the product.”

Why Use a Hybrid System?

You might be thinking: Why not go entirely Agile, since that has proven to be more adaptive, innovative, and efficient? Conversely, why not just continue working in Waterfall if that’s what an organization is used to?

Sometimes, there are barriers to going pure Agile, especially in highly regulated industries where products are subject to approvals from outside organizations that require documentation and rigorous timetables. For Juan Vilmaux, a Córdoba, Argentina-based project manager who is also on Toptal’s Project Management screening team, Waterfall is useful in projects where risk is a primary driver, such as in fields where companies face audits. (Since risk analysis needs to be done prior to project planning in these industries, changing something in that analysis would require a new plan.) “I was working for a company that ran clinical trials, and audits are crazy there,” he says. “So you have to go through several processes that are defined by external authorities like the FDA. If you work in Agile, you are constantly adjusting your scope or backlog—reprioritizing it—and that can interfere with these audits.”

David Machiels, a Brussels, Belgium-based Toptal project manager, says you have to be careful about the timing of releases in jobs that demand privacy protections. He led a hybrid team on the development of an identity management platform in Microsoft Azure Active Directory for a European banking group. It used Agile on some development steps, but since banks need to be protective of privacy data and are loath to put that information in the cloud, his team provisioned the system on a local server. “First, you need that on-premise implementation to be done,” he says. “You also need the cloud implementation to be done. Then you can start doing the connection between the two. There are a lot of steps that you have to do in a certain order.”

Most of the project managers we spoke to for this article have worked on hybrid projects for clients in the financial industry due to that industry’s inherent combination of strict regulations and the need to secure data. Grant Schuleman, a Johannesburg, South Africa-based Toptal project manager, has worked in financial services, banking, and stock exchanges. He says he delivered an equities trading engine and a derivatives trading engine “where there was a lot of integration related to master data and lots of regulatory requirements.”

Needing to adhere to a series of set steps lends itself to a Waterfall approach, but you can improve that process by incorporating Agile. Large epics can be broken into user stories for more flexible development, but then completed epics can be released on a longer timeframe. “Sometimes you have what I call a Big Bang delivery,” says Schuleman. The team deploys incrementally to a user acceptance testing (UAT) environment, and once all the features are signed off in UAT you release to production as one large deployment. “And that could take a year, depending how big the project is,” he says.

On his largest program, Schuleman had 120 people working on 10 projects—some working in Waterfall, some Scrum, and some hybrid. He also ran a “Scrum of Scrums” every other week to ensure that all of the smaller teams were aligned for the next series of sprints and working at complementary paces.

The Hardest Part of Going Hybrid

Implementing an Agile-Waterfall hybrid system can be a very situationally dependent endeavor. As the guide for this process, the project manager must find the right mix of methodologies that is appropriate for the product, team, and people who are going to be using them. “If you are trying to apply some of those methodologies as is, without any tailoring,” Anicin says, “it’s going to be a 100% failure.”

Schuleman tried to integrate Agile processes into an update of a legacy application but had to revert to pure Waterfall in the middle of the project. The experiment failed because it wasn’t embraced by the developers, all of whom were used to working in Waterfall and didn’t understand why the work was broken into epics that were further broken into user stories. They would see a user story in a sprint, Schuleman says, but couldn’t understand why “there are 10 other user stories also related to this application that are not yet in the scope.” They wanted to work on everything at once.

So what is the most consequential factor in the success of a hybrid system? The people. Some people are open to change and excited to try new things; some aren’t. When they aren’t, oftentimes their resistance boils down to a lack of understanding. The team members and management “don’t necessarily understand project management beyond a superficial level,” Stewart says. “They’ve been using Waterfall and they know there’s an Agile buzzword.” Since they’re not well-versed in the new process, they might want to do what they’ve done in the past.

And hybrid isn’t for everyone. Some project managers find that bridging opposing methodologies causes more problems than it solves. “It’s usually not a good approach to go hybrid,” Vilmaux says. “You’re increasing your chances to fail because you’re getting the worst parts of both worlds. You restrict Agile, but the nature of Agile is to embrace changes and to be flexible. You start losing all of that if you work within a Waterfall environment that works best in a linear fashion—fixed and deterministic—and where changes aren’t impossible but can be very costly. In adding Agile, you start pushing nonlinear things within that Waterfall world.”

That said, despite its complexity, hybrid done right can certainly pay off. Anicin recently led a successful hybrid project for the IFC (a member of the World Bank Group) in Republika Srpska, Bosnia and Herzegovina. “As an IFC project,” he says, “it was plan-driven—completely—but we agreed that we were going to apply a blended approach. The requirements and specifications—everything—were highly detailed, but our teams worked in Agile.”

The beneficiary of this process was the Republika Srpska government, which got a better product for a lower cost on a quicker timeline. There were “many organizations involved—government organizations, completely different organizations,” Anicin says. “It was hard, but it worked.”

Using Hybrid As an Agile Upgrade

With every industry undergoing a digital transformation these days, more companies are becoming curious about Agile. “HR is going a little bit Agile,” Stewart says. “Also, I know of a police organization in the Netherlands that uses task boards to clean up the backlog of crimes.”

Even if the company that you’re working with isn’t ready to jump in with both feet, a gradual introduction to Agile can pay dividends over time. One major reason for this is that Agile is very good at dealing with uncertainty. “Agile, in my mind, is far better when there are unknowns,” Schuleman says, “and 99% of the time there are unknowns.”

Another advantage is Agile’s inherent transparency. “I prefer Agile because I can see the changes,” he says. “With Waterfall, there can be a lot of smoke and mirrors: ‘We’re 20% complete,’ ‘We’re 30% complete,’ but then you’re stuck on 80% for another eight months. It’s far easier to hide problems. With Agile, you’re having daily standups, and if a user story is sitting there for longer than it should be, it’s easy to say, ‘This thing’s not moving; what’s the problem?’”

Incorporating Agile into any system can save money and time, all while delivering value that is more in line with what the customer actually needs. When a project fails, Stewart asks if that’s because it should have been more Agile. “I bet in 35% to 40% of IT projects that fail every year,” he says, “a good chunk of them that are Waterfall should be Agile—and that’s just not happening.”

Incorporating Agile slowly through a hybrid system can provide certain advantages, especially when there are aspects of a Waterfall way of thinking that can provide benefits to your team. For Anicin, the focus of a good hybrid system “is on the discovery part. When we are talking about the blended approach, we are providing a much more detailed product backlog”—much more extensive than he would create for a pure Agile project. In a hybrid project, he uses this more detailed backlog to give his team a more Waterfall-esque detailed perspective on the longer-term expectations for the finished product. Anicin then onboards his team “to the product, not only to the project,” he says. “I’m expecting the whole team to understand the product details because they need to have this product ownership, which is so important.”

Your company and your team might not be ready to go pure Agile, but at the very least, you can realize considerable benefits from adding Agile practices like daily standups and shorter, more frequent delivery deadlines. If you’re rigorous, smart, and careful in how you implement it, a hybrid system could be just what you need to upgrade your project.