While agile methods have become popular in commercial software development organizations, the engineering disciplines needed to apply agility to mission-critical software-reliant systems are not as well defined or practiced. To help bridge this gap, the SEI recently hosted the Agile Research Forum, which brought together researchers and practitioners from around the world to discuss when and how to best apply agile methods in the mission-critical environments found in government and many industries. This blog posting, the first in a multi-part series, highlights key ideas and issues associated with applying agile methods to address the challenges of complexity, exacting regulations, and schedule pressures that were presented during the forum.

Carleton's Forum Introduction

When introducing the forum, Anita Carleton, director of the SEI's Software Engineering Process Management Program, summarized how agile methods can provide customer value sooner and enable organizations to respond to change more quickly. Carleton started by highlighting the four key tenets presented in the Agile Manifesto, which forms the foundation for many agile methods, such as Scrum and Kanban:

• people over processes and tools
• working software over comprehensive documentation
• customer collaboration over contract negotiations
• responding to change over following a plan

Carleton explained that agility means the ability to move quickly and easily, the ability to think and reason quickly, or to possess intellectual acuity. "In a business context this is the definition of agile that matters," Carleton said. Agility in business means having an organization that moves, thinks, and responds quickly to change, not only in the short term, but over the lifetime of the system product or relationship. Agility is the ability to provide customer value sooner, to align development tempos with operational tempos, and to turn fast response into the business competitive advantage.

While agile methods have become popular with many information technology (IT) professionals as a means to replace perceived top-down bureaucracy with greater self-discipline and team-discipline, Carleton noted in her presentation that the SEI emphasizes measurable performance value for DoD programs and contractors who apply agile practices. Agility is not a capability you achieve by accident. Like agility in sports, which requires teamwork, strategy, training, management, and discipline, achieving agility in a software development organization is no different.

Carleton explained that the SEI's research and transition efforts have concentrated in recent years on enhancing lightweight approaches and processes to address what's needed and required by our sponsors, partners, customers, and other stakeholders. Meeting these needs involves scaling up agile methods to the mission-critical software-reliant systems common in the Department of Defense (DoD), as well as mission-critical programs in other domains, such as finance, energy, telecommunications, space exploration, and aviation.

Carleton outlined the following areas where SEI work is focusing on applying agile methods at-scale:

• defining and evaluating practical guidance for DoD project managers, systems engineers, and contracting officers who are considering the adoption of agile methods

• defining metrics that can be used to measure and appraise the performance and success of programs that apply agile methods

• working with DoD acquisition programs to pilot and roll-out agile methods in combat system development environments

• developing an architecture-focused measurement framework for managing technical debt in agile projects

• formulating a decision making framework for reducing integration risk with agile methods

• applying agile principles in strategic planning processes

Takai's Keynote Presentation

In her keynote presentation during the forum, Teresa M. Takai, chief information officer (CIO) for the DoD, discussed how agile methods have been introduced into the DoD software acquisition and development environment. As the DoD CIO, Takai provides IT support for 2 million individuals across the globe: 1 million on the civilian side and 1 million on military side. Providing IT support for the DoD means delivering reliable computing and communications capability for warfighters, regardless of their location. IT is an essential part of what the DoD does to enable service men and women to perform their duties.

Challenges that Motivate DoD Agility

The DoD faces many challenges--including budget pressures and rapid technology insertion--that motivate the need for agile methods, Takai said during her keynote. These challenges have resulted in two realities for IT development:

• The DoD must be a good custodian of technology dollars, Takai commented during the forum, pointing out that the DoD spends at least $38 billion a year on IT.
• The DoD also needs to speed up IT delivery. Men and women hired by the DoD now expect to do their jobs using their smart phones, the same way that they do in their private life, Takai explained. Long development cycles don't fit with user expectations. In the DoD, it can take up to 81 months to acquire or develop a new technology. "Typically, slow acquisition time has been blamed on the acquisition process, but that's not always fair," Takai said. The challenge for the DoD is that the acquisition process encompasses front-end requirements gathering, recruiting industry partners, and, testing and implementation. The mandate for the DoD to change is enormous.

Agile practices are not just a methodology, they involve a cultural change in the way business is conducted. Takai suggested to the audience that cultural change is the hardest part of adopting agile methods in the DoD. Nearly 33 percent of DoD IT programs are canceled during development because, as programs move through a process taking 81 months, they realize they can't deliver the capability they had intended to deliver. Over 60 percent of DoD IT programs are late and/or over budget. Larger IT projects have a much larger risk of over budget and under delivery.

Cultural and Process Changes Need to Support DoD Agility

As part of the DoD IT acquisition reform effort, Takai explained that DoD leaders are examining how to take agile best practices, continue to educate the IT technology workforce on the meaning of these best practices, and ensure that all involved understand the overall DoD culture to ensure IT developers can apply agile methods effectively. This cultural change involves enhancements to established DoD policies and practices. The DoD is part of a larger government-wide effort that, under the US CIO, published a 25-point plan, a portion of which prescribed transitioning to agile methods, as elaborated in Takai's 10-point plan for IT modernization in the DoD.

Some segments within the DoD have already begun transitioning to agile methods, Takai said. Based on these experiences, the DoD has been establishing the framework and a handbook that program managers can to guide their implementations of agile methods within its organizations. One of the DoD's strategies has been to share best practices. An important best practice has involved establishing a governance process that promotes the successful adoption of agile IT methods.

DoD organizations are accustomed to developing DoD specific solutions, such as radio or weapons systems, that involve rigorous processes. Unfortunately, with this approach it's hard for program managers to decompose software systems into smaller, more deliverable chunks necessary for agile development. "The DoD can no longer simply write and sign off on requirements and then turn it over to acquisition to manage delivery," Takai said. The DoD must find a better way to involve the user through the development process. For the DoD that's not always been the norm, so making that move involves cultural changes.

Takai said that as the DoD considers large-scale IT development projects (some in the billion-dollar range), leaders need to learn how divide them into chunks effectively. This decomposition process should start by specifically examining the ongoing requirements process, involving the user, and ensuring a much stronger governance process. The DoD must also examine how to manage agile development from a risk-mitigation standpoint.

One challenge of traditional waterfall methods is that the focus is often on avoiding risk. Risk avoidance is not what IT is about anymore, Takai said. Implementing agile methods will enable the DoD to mitigate risk and make the changes needed after a small increment of delivery, and then build on that concept to reach next stages of delivery. That approach is a tough concept for the DoD, which has historically focused on ensuring that requirements and processes remain iron clad, with minimal risk involved. Such an approach eliminates the ability to bring in innovative technologies, as well the ability to implement industry best practices. That's not the way to move forward, she said, especially in an era of austerity in the DoD budget.

What's Next

Our next posts in this series will summarize discussions of four SEI researchers, including myself, at the Agile Research Forum who examined aspects of applying agile methods at-scale in mission-critical development environments:

• Mary Ann Lapham highlighted the importance of collaboration with end users, as well as among cross-functional teams, to apply agile approaches to DoD acquisition programs successfully at-scale. She noted that effective agile DoD teams are flexible, experienced, and able to work fluidly between disciplines.

• Ipek Ozkaya discussed the use of strategic, intentional decisions to incur architectural technical debt. The technical debt metaphor describes the tradeoff between taking shortcuts in software development to speed up product delivery and slower--but less risky--software development.

• James Over noted that lack of teamwork can critically impede agility. He advocated, among other principles, the building of self-managed teams, planning and measuring project process, designing before building, and making quality the top priority for achieving agility at-scale.

• Finally, I wrapped up the forum with a discussion on the importance of applying agile methods to crucial common operating platform environments (COPEs) at the DoD. I explained how agile methods can encourage more effective collaboration between users, developers, testers, and certifiers to help the DoD successfully build integrated, interoperable software systems.

In addition to providing you weekly updates of the latest research from our technologists, the SEI blog has also become a catalyst for sparking thoughtful discussions on the latest challenges facing commercial and DoD organizations. We therefore look forward to hearing your thoughts on applying agile at-scale in the comments section below.

Additional Resources

The slides and recordings from the SEI Agile Research Forum can be accessed at
www.sei.cmu.edu/go/agile-research-forum/.