Over the past several years, the SEI has explored the use of Agile methods in DoD environments, focusing on both if and when they are suitable and how to use them most effectively when they are suitable. Our research has approached the topic of Agile methods both from an acquisition and a technical perspective. Stephany Bellomo described some of our experiences in previous blog posts What is Agile? and Building a Foundation for Agile. This post summarizes a project the SEI has undertaken to review and study Agile approaches, with the goal of developing guidance for their effective application in DoD environments.

The SEI's Agile project began in 2009 in response to our recognition of the growing awareness that Agile methods help alleviate key challenges facing the DoD, such as providing competitive capabilities to warfighters in a timely manner that minimizes collateral damage and loss of lives and property. We also observed an emerging consensus that Agile methods can be applied to create systems who functionality and quality attributes can be adapted more readily over time, which may help reduce total ownership costs over long acquisition program lifecycles. Within this context, the primary activities of our project included reviewing relevant literature, interviewing programs that are using or have used Agile methods, identifying criteria to be used to determine if a program is a candidate for Agile methods and what risks exist for implementing the Agile methods, and creating guidelines to be used in implementing agile methods.

We initially focused on the question of whether Agile could be used in DoD acquisition programs, which historically follow the DoD 5000 series of guidelines that have been associated with so-called waterfall methods. An early finding of our project was that no prohibitions preclude the use of Agile in the DoD 5000 series. This result is important since some skeptics have asserted that Agile is not suited for the DoD due to inherent conflicts between Agile methods and DoD policies and regulations.

Given that there is no one size fits all Agile method, however, we also found that implementations of Agile methods must be tailored to fit the situation and context. In other words, Agile is not a silver bullet. Transitioning DoD systems--and their associated socio-technical ecosystems--to Agile therefore requires considerable work from DoD agencies and the defense industry base, and is not without hurdles.

For example, we found that adapting Agile into the DoD acquisition life cycle presents unique challenges and opportunities. The challenges are in meeting existing DoD milestone and regulatory criteria when there is little if any guidance available on how to do so when using Agile methods. The opportunities, however, provide the capability to accomplish development with frequent results. Other hurdles identified by DoD programs we interviewed include

• providing the right team environment allowing access to end users
• determining how to train and coach the government staff
• instituting suitable oversight methods
• adapting rewards and incentives to the Agile environment
• adjusting to a different team structure

These hurdles are due to a different approach than business as usual and a general lack of specific training and guidance on Agile concepts and approaches within the DoD. Overcoming these hurdles requires changes to the waterfall-centric organizational culture that is common within DoD acquisition programs. These culture changes also require mindset changes because the underlying paradigm for implementing Agile methods is different from that used for the waterfall method.

After studying the topics described above to gain a preliminary understanding of the use of Agile within the DoD, we then studied other management, acquisition, and technical topics, as described below.

Agile Management and Acquisition Topics

Our project also is studying the following management and acquisition topics relevant to the effective adoption of Agile methods in the DoD:

• Being Agile in the DoD. Agile methods provide promising techniques for streamlining the acquisition process for systems within the DoD. To meet the challenges of adopting Agile methods, however, DoD program management offices must take specific actions to assist in Agile adoption and even enable it. For example, to ensure successful Agile adoption, DoD organizations must plan for it, train for it, anticipate changes in their environments and business models to ensure the benefits of Agile become a reality.
• Managing and contracting for Agile programs. Managing large-scale, complex software-reliant systems is always hard, but management in Agile programs takes on some added dimensions. For example, program managers not only must be leaders, they must also be coaches, expeditors, and champions. If they do not personally perform these roles, someone in their organizations must responsible for them. These additional roles are needed due to the paradigm associated with using Agile methods and the lack of any significant experience by current DoD personnel in that arena. A particular management concern is the selection and implementation of appropriate contracting vehicles to support the types of practices that successful Agile projects exhibit.
• Technical milestone reviews. One sticking point in employing Agile methods is how to accommodate large capstone events, such as the preliminary design review (PDR), critical design review (CDR), and others. While many concerns exist in this area, it's important to focus on the purpose of holding these reviews in the first place: to evaluate progress on and/or review specific aspects of the proposed software solution. Expectations and criteria must therefore reflect the level and type of documentation that is acceptable for the milestone, which is no different from business as usual. The key, however, is to define the level and type of documentation required for the specific program while working within an Agile environment.
• Estimating in DoD Agile acquisition. Estimation done on Agile projects is typically not the same as the traditional methods used on legacy systems within DoD. Traditional methods tend to focus on estimating details up front; these details are then modified as more information is obtained. In contrast, Agile estimates are often "just-in-time," with high-level estimates that are refined to create detailed estimates as knowledge of the requirements matures. Some tools within the traditional estimation community are now adding modules to address Agile estimation.
• Moving toward adopting Agile practices. Change is hard--especially for large DoD ecosystems--and understanding the scope of changes is essential. Organizational change methods must be employed to help DoD organizations successfully adapt to applying Agile. There are multiple adoption factors (such as business strategy, reward system, sponsorship, values, skills, structure, history, and work practices) that must all be addressed. Change-management best practices include understanding the adopter population, understanding the cycle of change, understanding the adoption risks, and building transition mechanisms to mitigate adoption risks. Organizations we studied that had successfully adopted Agile methods typically achieved the following goals:

• Found and nurtured good sponsors for Agile adoption
• Understood the adoption population they were dealing with
• Conducted a readiness assessment that addressed organizational and cultural issues
• Analyzed what adoption support mechanisms were needed for a particular context and built or acquired them before proceeding too far into an Agile adoption

SEI Agile work continues and the following additional documents are--or will soon be--available:

• Case Study of Successful Use of Agile Methods in DoD: Patriot Excalibur 2011-TN-019
• Agile Methods: Changing the Viewpoint of Government Technical Evaluation 2011-TN-026
• A Closer Look at 804: A Summary of Considerations for DoD Program Managers 2011 SR-015
• "DoD Agile Adoption--Necessary Considerations, Concerns, and Changes" in the Jan/Feb 2012 issue of CrossTalk

In addition, the SEI has created an Agile Collaboration Group to advise, review, enhance, and validate SEI acquisition work. The SEI is working with this group to create, calibrate, and validate a contingency model that will help acquisition professionals determine when to use Agile techniques, as well as how to identify potential risks if Agile methods are adopted. We are also creating guidelines that summarize best practices and instruct users of Agile methods on how to apply these methods effectively in DoD environments.

Agile Technical Topics

Agile software development has historically succeeded in small-scale (largely IT-based) commercial environments due largely to its easy-to-apply practices for tracking project status and allocating the development resources to those activities that deliver the most potential customer value. A key technical challenge for DoD projects, however, involves balancing the short-term and long-term needs. In particular, the cliché "You aren't gonna need it (YAGNI)" is a principle in eXtreme Programming (XP) that implies developers should not add functionality until it is necessary, thereby eliminating a considerable amount of unused code in a system. The YAGNI principle rarely seems to apply, however, in large-scale DoD environments, where systems must operate for decades with continual flux with respect to evolving requirements, technology upgrades, new partners, and different contractors.

The SEI is conducting the following technical work on successfully creating and applying Agile methods for the DoD:

• Agile at scale. This work focuses on providing methods and techniques for applying Agile software development practices for large-scale DoD programs, with improved visibility into the release plan and the quality of the system. One of our activities to address the use of Agile development at scale was conducting a field study with organizations that deal with the challenges of Agile and architecture practices at scale. Based on our observations, we developed a readiness, best practices, and risk analysis technique. Striking the proper balance between developing the system and the architecture in an agile manner while providing enhancement agility for maintaining the system is key to success. We observed that tactics that help organizations to succeed within an Agile environment include paying close attention to architecture-centric practices where a balance of feature development and architecture development is achieved.
• Technical debt. Our work in technical debt analysis again focuses on architecture and looks at strategically incurring technical debt (such as applying architectural short-cuts) to improve agility in the short-term. This work focuses on developing techniques to monitor and respond to emerging rework, as well as the need to refactor or rearchitect the system to pay back the debt. The need to refactor or rearchitect DoD systems arises in several ways. For instance, system quality degradation, such as unacceptable end-to-end performance, might require refactoring. Such quality degradation-related rework can appear if the development teams focused solely on feature-oriented decomposition of the system to deliver features at early iterations, but didn't provide the necessary architecture for the infrastructure in a timely manner. Refactoring would require the restructuring of the existing body of code to alter its internal structure (architecture) but not change its external behavior to address the decrease in quality.
• Modeling decision impact on agile development. Our work also provides guidance and techniques that enhance the applicability of mainstream Agile and lean software development methods to DoD stakeholders by balancing their acquisition and technical needs. In a recently started project, for example, we are investigating acquisition and architecture activities during the pre-engineering manufacturing and development phase of the acquisition lifecycle. This work closely examines modeling decision dependencies and analyzes their impact on the ability to conduct effective Agile system development. This work targets the perspective of reducing integration risks in large-scale DoD systems.

In summary, our projects have found that Agile methods can indeed provide both tactical and strategic benefits in the DoD. The tactical benefits of lower cost, on-time delivery, and increasing quality are clearly important as the DoD places a growing emphasis on greater efficiency in its acquisition processes. The strategic benefits of responsiveness and more rapid adaptability to the current situation, however, may be of even greater value in today's world, where the DoD must get results faster and be better aligned with changing needs to prepare for an uncertain future. As our work progresses, we will periodically post our progress, ask questions, and request feedback. If you have any questions or feedback on the current work, please post in the comments below.

Additional Resources

Please see the following SEI technical reports and notes for more on Agile development:
Considerations for Using Agile in DoD Acquisition
Agile Methods: Selected DoD Management and Acquisition Concerns
Documenting Software Architectures in an Agile World
CMMI or Agile: Why not Embrace Both!
Incorporating Security Quality Requirements Engineering (SQUARE) into Standard Lifecycle Models
Secure Software Development Life Cycle Processes--A Technology Scouting Report
Integrating Software-Architecture-Centric Methods into Extreme Programming (XP)