While agile methods have become popular in commercial software development organizations, the engineering disciplines needed to apply agility to mission-critical, software-reliant systems are not as well defined or practiced. To help bridge this gap, the SEI recently hosted the Agile Research Forum. The event brought together researchers and practitioners from around the world to discuss when and how to best apply agile methods in mission-critical environments found in government and many industries.

This blog posting, the fourth installment in a multi-part series highlighting research presented during the forum, summarizes a talk by James Over, manager of the Team Software Process (TSP)initiative, who advocated the building of self-managed teams, planning and measuring project process, designing before building, and making quality the top priority, among other principles associated with applying agile methods at-scale.

Over's Talk on Balancing Agility and Discipline

In his opening comments to the audience, Over shared his views on agility and discipline and stressed the importance of finding a balance between the two. Over said that his presentation and research on combining agility and discipline is based on his work with software teams and software projects, although it is applicable to other fields.

One of the reasons that agile methods are so popular today is their ability to respond to change. As evidence of this popularity, Over pointed out that about 40 percent of software developers use one or more agile methods, compared with 13 percent that only use traditional methods, according to a 2010 Dr. Dobbs survey on trends among global developers.

One reason that agile methods have become so popular is that the pace of change is accelerating. Organizations are seeking solutions that will allow them to become more responsive to change. Agile methods provide some key parts of that capability, Over said.
Balance is key for organizations seeking to implement agile methods. While organizations work to improve agility, they must do so in a disciplined way. Discipline is particularly important for organizations like DoD acquisition programs and other federal agencies developing large, mission-critical software-reliant systems at scale.

It's important for organizations to understand what is meant by agility. While several definitions exist, Over stated one he likes: Agility is responding rapidly and efficiently to change with consistency. An agile business, he told the audience, should be able to respond quickly to change, make decisions quickly, and respond quickly to customers' needs every single time, not just occasionally.

What Does Agile Look Like

Many software organizations claim to be agile because they follow agile principles, but they often lack an understanding of what it means to be agile. To achieve a greater understanding of agile methods, Over recommended that organizations measure their agility and evaluate their success along the following factors:

• Response time. Organizations should assess how quickly they respond to a customer's needs. What sort of experiences are users having with the software they are producing in terms of response time?
• Efficiency. Organizations should consider their ability to deliver software. Can they produce projects with the desired balance between cost and quality? Are processes performing efficiently? Can organizations respond to change quickly and efficiently or do costs balloon out of control when changes are made to the content or requirements of the system?
• Consistency. Does every customer have the same experience? When customers interact with an organization, are they always seeing the same response time, the same types of efficiency, and the same types of behavior on each application?

Impediments to Agility in Software

In 2010, the authors of the Agile Manifesto reunited to hold a retrospective on Agile. Examining the state of the practice in the last decade, the authors identified 10 impediments to achieving agility. From that list, Over identified the following five impediments that he deemed critical for organizations to overcome when they seek to making agile methods work in practice:

• Lack of a ready backlog, which is the list of features that developers prioritize to build the software, is a serious concern. The manifesto authors found that 80 percent of teams had a ready backlog, but within that backlog, only 10 percent of items were ready for implementation. As a consequence, delays would occur in projects because the backlog was not prepared.
• Lack of being "done" at the end of a sprint means that as the project is nearing the end of the sprint and declaring the sprint completed, some work items are being deferred or skipped, which often causes delays. The most common cause is a poorly implemented practice or a deferred practice. For example, skipping unit testing can cause delays later in the project, as well as increase cost by at least a factor of 2.
• Lack of teamwork surfaces when the team fails to come together and operate as a team and instead focuses on its individual needs. In this type of setting, developers focus on their own user features or stories and don't pay attention to what the team is doing. They attend their daily standups, but fail to report problems that might affect the rest of the project. When team gets to the end of a sprint and discovers that there is still a lot of work to do, therefore, sprint failure and project delay will result.
• Lack of good design stemming from organizational structures that affects the kind of designs that software teams produce. For example, an inflexible hierarchical structure often results in inflexible hierarchical designs, which in turn can yield code that is brittle and hard to use, excessively expensive, and prone to high failure rates.
• Tolerating defects is an unfortunate reality for many teams as they near the end of a sprint. They are out of time and deferring defects to the end of a sprint. Teams in this situation often take a set of unit tests or other issues and defer them to later sprints. When issues are deferred (which is a form of technical debt, as discussed by Ipek Ozkaya in her Agile Research Forum presentation), the result is increased in cost and higher failure rates later in the lifecycle because the defects aren't being addressed as they are discovered.

Avoiding the Impediments by Balancing Agility and Discipline

Over next identified work that he and other SEI researchers have conducted to remedy the impediments to agility described above. Over's work with software teams has focused on identifying a set of principles that teams should adopt to improve agility and response time. Over highlighted the following five principles that help address impediments identified by the Agile community in their retrospective:

• Build high-performance teams. Software is knowledge work. Build self-managed teams that make their own plans, negotiate their commitments, and know the project status precisely.
• Plan every project. Never make a commitment without a plan. Use historical data. If the plan doesn't fit the work, fix the plan. Change is not free.
• Use a measured process to track progress. To measure the work, define its measures. To measure the process, define its steps. If the process doesn't fit the work, fix the process. Tracking progress via a measured process need not require a complicated solution given the appropriate set of tools and a codified method for applying the tools effectively.
• Design before you implement. Design is critical since it informs the software implementation. Good designs produce less code and simplify the downstream evolution of the code. Design what you know and explore the rest. When you know enough, finish the design and then implement it. To be clear, this principle isn't espousing "big upfront design" (BUFD). It is BUFD without the BUF, or just D. Produce enough design to build the implementation. If the design is still too challenging, explore the problem first via prototyping to reduce design risk. Applying this principle effectively depends on various factors, including the size and complexity of the application, as well as familiarity with the problem, domain, methods, and tools.
• Make quality software the top priority. Defects are inevitable. Poor quality wastes resources. The sooner a developer fixes a problem, the better. As Jeff Sutherland stated in his retrospective, tolerating defects, ignoring pair programming or code review practices, and inadequate unit testing will substantially reduce team velocity. Continuous integration tools and automated testing are also important, but testing alone is insufficient because poor quality software will always cost more to produce because finding and fixing defects is still expensive, and the longer you wait to fix them the greater the cost of repair.

In summary, Over told the audience that in working on 20 different projects in 13 organizations to implement these disciplined agile principles, SEI researchers found that organizations delivered more functionality than originally planned or finished ahead of schedule. Among the other benefits, Over stated, was that projects realized test costs of less than 7 percent of the total cost with an average cost of quality of only 17 percent. Also, the projects delivered software with an average of only six defects in 100,000 lines of new and modified code.

Finally, discipline is the key to agility, Over explained, adding that agility can only be achieved when everyone in an organization acts professionally and uses a disciplined, measured approach to their work.

What's Ahead

The first posting in this series summarized discussions by Anita Carleton, director of the SEI's Software Engineering Process Management Program, and Teri Takai, chief information officer for the DoD. Carleton provided an overview of the forum and discussed areas where SEI work is focused on applying Agile methods at-scale. Takai then discussed how Agile methods have been introduced into the DoD software acquisition and development environment. The second posting summarized discussions by Mary Ann Lapham, a senior researcher in the SEI's Acquisition Support Program, who highlighted the importance of collaboration with end users, as well as among cross-functional teams, to facilitate the adoption of Agile approaches into DoD acquisition programs. The third posting highlighted the forum presentation by Ipek Ozkaya, a senior researcher in the SEI's Research, Technology, and System Solutions Program, who discussed the use of strategic, intentional decisions to incur architectural technical debt. The technical debt metaphor describes the tradeoff between taking shortcuts in software development to speed up product delivery and slower--but less risky--software development.

In the next--and final--posting in this series I will summarize my presentation at the forum on the importance of applying agile methods to common operating platform environments (COPEs) that have become increasingly important for the DoD. I will explain how these methods can encourage more effective collaboration between users, developers, testers, and certifiers to help the DoD successfully build more integrated, interoperable, and affordable software systems.

We look forward to hearing your thoughts on applying agile at-scale in the comments section below.

Additional Resources

To learn more about the Team Software Process (TSP), please visit www.sei.cmu.edu/tsp.

The slides and recordings from the SEI Agile Research Forum can be accessed at
www.sei.cmu.edu/go/agile-research-forum.