Agile Documentation: Balancing Speed and Knowledge Retention
The various documents, artifacts, and processes related to their generation are some of the main symbols of the Waterfall model. Borrowing from Lean, Agile considers a lot of documentation as “waste” that needs to be eradicated in order to streamline the development lifecycle.
For many project managers, it’s not a big stretch to understand how the Waterfall phases are turned into sprints—the same work is accomplished; it’s just organized in a different way. However, the removal of most documentation is a harder pill to swallow as it underlines a completely different way of working. It requires loosening the reins of control, embracing the unknown, and empowering the delivery team to make decisions on the spot.
The Traditional Approach to Documentation Is Being Challenged
In the Waterfall methodology, a lot of time is spent on documenting project requirements and detailing solutions. This process works when the requirements are completely clear and we are certain that nothing will change from what has been captured and baselined. Yet the experiences of most companies in the last few decades have shown that this is almost never true. In today’s world, the pace of change is so dynamic that the needs of the client change considerably by the time we complete the documentation phase.
Agile’s focus is on getting things done and adding value to stakeholders. It is built in such a way that the model discourages working on the peripherals and on activities that do not directly and immediately add value to the client.
Documentation in Waterfall vs. Agile
Each company has a different level of documentation, which can differ even on a project level. But here is what typical documentation procedures look like in Waterfall and Agile projects:
|Documentation is mandatory for most of the time. No work progresses unless the documentation is complete.||Only basic documentation to understand just enough to get started on the work is encouraged.|
|Documents undergo a lengthy review process and must be approved by multiple parties.||There is no formal review and approval process and the project manager is the key decision-maker.|
|Standardized templates need to be followed.||There are no formal templates for documentation; best practices are used instead.|
|Various types of documents are required at different phases: project charter, vision statement, business requirement document, functional and non-functional requirements, high-level design (HLD) and low-level design (LLD) documents, etc.||Only the documents that are needed to deliver functionality in the upcoming sprint are required.|
|Changes to documentation are hard because all documents are intertwined.||Changes to documentation are much easier.|
|A system or process to manage a large number of documents is needed.||Documentation is minimal, hence easy to manage.|
The Case for Documentation
Waterfall promotes a much stricter approach to documentation, which might seem excessive. But before we dismiss this as “waste,” here are several benefits are of having strong documentation procedures.
Opportunity for Strategic Thinking
Those who fail to plan, plan to fail. Documentation forces a project manager to sit down and think things through then come up with the best solutions. People sometimes misinterpret the Agile value of working software over comprehensive documentation to mean that no documentation is required. They then rush off to the market, a move Paul Adams, VP of Product at Intercom, describes as throwing stuff at the wall and seeing what sticks. Designing solutions, creating plans, deliberating actions—these activities create value by saving time on not developing every single feature idea that comes to mind.
UX and Functional Consistency
As companies grow from a few founders to hundreds or thousands of employees, many different teams start working on the same or related products. Team A might think that what they’re working on isn’t related to what team B is working on, but to the end-user, it is all the same product. Instead of cross-functional teams doing their own thing, clear documentation on the user experience and functional levels avoids a disjointed user flow.
Documentation Can Be Converted to User Guides
In Waterfall, a lot of time is spent detailing the solutions and how they’ll be used. Pictures of high-fidelity designs are created for front-end developers. All of these assets require less work to be converted into internal or external user guides than creating them from scratch.
How Agile Reduces Documentation Needs
A factor that repeatedly comes up as an excuse to mandate documentation is employee turnover. Managers fear losing institutional knowledge when people leave and new ones join to replace them. How will they know what’s been implemented and how it works? How long will they take to catch up? Will the current team have the bandwidth to handhold new team members?
The hope is that good documentation will get new employees who work mostly independently quickly up to speed. However, Agile inherently reduces the need for documentation through collaboration techniques that, at the same time, reduce the time for onboarding. Here are a few ways Agile reduces the need for documentation.
Regular Interaction Between Product Teams and Agile Team Members
The Agile Manifesto promotes “individuals and interactions over processes and tools.” Since requirements tend to change during a project and new ideas arise, Agile ensures clarification of requirements directly from the source instead of depending on written artifacts that need constant updating.
Grooming and Planning Compartmentalizes Tasks
Backlog grooming and sprint planning break features down into specific, implementable parts that are easily understood and can be worked on independently. This creates an opportunity for new hires to be productive early on, while not yet fully understanding the big picture of the whole project.
User Stories Provide Efficient Documentation
The simple format of user stories allows project managers to capture the bare minimum requirements that create a shared understanding among all team members. Even if a user story doesn’t make it into a sprint, the waste of creating this documentation artifact is very low. As user stories move into sprints, they can be fleshed out and supplemented with other required information such as wireframes, designs, acceptance criteria, etc. This process gives a very efficient documentation delivery that’s highly disposable and produced at the most appropriate development stages.
Reduced Need for Code Documentation
Techniques such as pair programming and code review create constant opportunities to disseminate technical knowledge throughout the team, especially to new team members. The constant feedback leads to a shared understanding that also has the flexibility to adapt to new circumstances instead of quickly becoming outdated in a document somewhere.
Daily standups, sprint reviews, and retrospectives create ample opportunities to resolve issues and make decisions face-to-face instead of relying on email and documents. The limited timeframe of all ceremonies ensures that only the most important information is prioritized instead of spending time documenting everything, even if it is likely never to be used.
All of the above directly or indirectly reduces documentation and prioritizes the delivery of project goals while ensuring that nothing is really lost by a lack of documentation.
Hybrid Approaches to Documentation
Some companies still prefer to have some documentation, even in an Agile setting. Agile is not prescriptive, because each project is different and has a unique set of circumstances that need to be addressed.
Below are some examples of how Agile can be combined with more time-intensive methods of documentation.
Combining UML and Agile
Consider using a standard modeling language such as Unified Modeling Language (UML) which is very structured and has defined entities to visualize a system. This helps to keep the content very simple and focused on what is needed and ensures minimal use of written language. Tools such as StarUML and Draw.io, among many others, are convenient options.
Code Documentation Generators
Another approach is to ensure that the code is a lot more readable by introducing more structured and detailed comments as part of the class details, method details, parameter usage, dependencies, and so on. There are many tools that automate the process of generating useful documents from source code and they are called documentation generators. They range from generic to programming language-specific ones.
Detailed Design and UX Documents
Defining requirements using wireframes, mockups, user flow diagrams, sequence diagrams, etc. helps to simplify the project flows as well as make it explicitly clear to the technical team what needs to be developed. Design documents are a great way to have a varying level of more rigid documentation. There are various wireframing and UX tools to choose from for these tasks.
Project Management Tools Automate Documentation
More powerful project management and related tools such as JIRA, Confluence, Asana, and Basecamp provide a way to hold all project-related information in one place. Tasks can be linked, tagged, nested, and assigned to different team members, who can then leave comments and report any problems. All of these actions, alongside the flexibility to adapt those tools, can create a substantial amount of documentation with little to no effort.
Moreover, historically, part of the documentation needs stems from reporting requirements. Stakeholders want access to team performance or other relevant metrics. Project management tools make it easy to automate custom dashboards and views that reflect project progress and link back to relevant documentation within the tool.
Documentation Management Is a Balancing Act
The creators of the Agile Manifesto write that they value “working software over comprehensive documentation.” However, they also add a disclaimer that “while there is value in the items on the right, [they] value the items on the left more.” Agile doesn’t suggest removing all documentation, because some documentation obviously provides value; it simply suggests that the priority should be on working software and adding documentation only as necessary depending on the circumstances of the project and without heavily impeding development progress.
Project managers have to achieve a balancing act between spending less time on documentation and spending more time on delivering working software and actually figuring out where some form of documentation is necessary for long-term success.
Understanding the basics
Does Agile have documentation?
Agile typically has some amount of documentation necessary to maintain a project stable. However, Agile promotes collaboration over documentation as a preferred way of sharing knowledge.
Does Agile mean no documentation?
Agile sometimes means no documentation if the team is capable of sharing knowledge effectively via other means. This is usually only possible in small teams and some documentation is inevitably introduced as the company grows to dozens or hundreds of people.
What are the documents that are produced in Agile methodology?
There are no predefined documents that have to be produced according to Agile. Teams have to decide for themselves what amount of documentation is required and would not impede development progress.
What is the main purpose of documentation?
The main purpose of documentation is to retain institutional knowledge and create a shared understanding between team members and any other relevant stakeholders.
What is a documentation process?
A documentation process is a set of predefined actions that explain how and what kinds of documents are created at various stages in a project.
Located in Bengaluru, Karnataka, India
Member since June 7, 2019
About the author
Vinod is a seasoned technical project manager who focuses on developing web, mobile, and digital products for global customers.