The DoD chief information officer (CIO)'s office has recently released a 10 Point Plan to reform DoD information technology (IT). Point number 4 is "Enable Agile IT." The key tenets of Agile IT include

• Emphasize incremental introduction of mature technology by delivering useful capability every 6 to 12 months to reduce risk through early validation to users.
• Require tight integration of users, developers, testers, and certifiers throughout the project life cycle to meet Agile IT's promise of rapid delivery in lieu of extensive up front planning.
• Leverage common development, test and production platforms and enterprise products to deliver IT capabilities faster, cheaper, and more interoperable, without redundant infrastructure and documentation.
• Establish a change-tolerant design environment enabled by discovery learning that promotes decisions based on facts rather than forecasts.

Program managers and acquisition executives are responding to this plan by applying industry practices, such as Agile methods. At the same time, related DoD guidelines encourage system development via a more open, modular software architectural style, such as loosely-coupled service-oriented architectures. The impact of these architectural decisions becomes more critical in an agile context because they promote or inhibit the ability to achieve agile goals, such as rapid feedback, responding to change, and team communication. Architectural decisions influence agile development properties, such as the length of an iteration, the allocation of stories to iterations, and the structure of the teams and their work assignments with respect to the architecture. What is needed, therefore, is research on eliciting and employing these properties and determining their relative importance in promoting rapid lifecycle development.

1. The fundamental early decisions made during the pre-engineering and manufacturing development (pre-EMD) phase have an impact throughout the lifecycle. Acquisition programs lack the techniques needed to elicit and preanalyze the implications of which fundamental architectural decisions will enable or inhibit their ability to adopt an agile lifecycle. For example, architectural decisions that relate to decomposition and dependencies influence teaming structure, the capability to rapidly and confidently deliver features, and the ability to rapidly integrate new components among other factors.
2. The implications of the early decisions often surface in the final stages of the lifecycle, downstream from development. For example, unexpected rework related to correcting integration defects. When these problems are discovered further downstream from where they were injected in the lifecycle, they are more expensive to correct and this often causes cost overruns and delays project completion.

Identifying Critical Properties

A primary objective of our research is identifying critical properties of Agile software development influenced by architectural decisions that reduce lifecycle time. Identifying these properties is an important first step to give stakeholders the tools they need to make informed decisions that manipulate the settings of the properties to control for better outcomes.

Our approach involves examining the implications of possible change scenarios. One such scenario examines the impact of introducing an emerging multi-core technology to the mission processor software of the Apache helicopter, which is a US Army/Boeing program. Applications are increasingly favoring multi-core processors, a single integrated computing element with two or more independent processors, called "cores," allowing for greater processing capacity. The scenario explores questions that include

• Will the multi-core technology change the architectural design (e.g., patterns, tactics, and architectural approaches)?
• If so, which architectural decisions will change?
• Are there engineering practices (such as Agile) that must be customized as a result of architectural decisions in support of a rapid lifecycle?
• How will continuous integration be ensured?
• How will team communication be impacted?

A key component of our work involves determining the critical decisions that will influence Agile practices. These decisions provide the rationale for how software and system architects design an architecture.

To expand our view, we will again collaborate with Philippe Kruchten, a professor of software engineering in the Department of Electrical and Computer Engineering of the University of British Columbia, who is active within Agile development and architecture decision making research communities. Another facet of our approach involves interviewing DoD programs and gathering data from members of the SEI Agile Collaboration Group.

Creating a Model

Rework must be considered early when reasoning about how to enable rapid lifecycle development. After we review the findings identified in the first phase of our work with collaborators, we will create an architectural decision model that allows software or systems architects to analyze the ramifications of their decisions. Based on our prior work, we anticipate that highly impactful architectural decisions will include

• Decisions about interfaces and how the parts of the system are connected.
• Decisions about structuring the systems to achieve quality attribute requirements. The impact of these decisions typically spans multiple areas within the system and is not localized within a single module.

We plan to use the Multiple Domain Matrix (MDM) to represent the decision model and to analyze the impact of architectural decisions on rapid lifecycle development. The MDM approach considers decision dependencies that provide visibility into how the ordering of decisions influences when development can be started and how changes propagate and may require rework of software elements. This approach will allow us to test our hypothesis that modeling architectural decisions during early stages of development (similar to pre-EMD) will reduce cycle time. Cycle time could be reduced upstream by enabling an earlier start of development, thus minimizing the time spent at the pre-EMD phase or reduced downstream by decreasing rework costs attributed to architectural decisions that affect integration.

Another component of our research will explore strategies for improving the relationship between architecture decision making and complex collaborative team interaction. A barrier that often arises is that decisions made by architects about partitioning the architecture are not aligned with the networks of agile teams at scale and for the kinds of systems relevant to the DoD. Another barrier is alignment with the teams that span the lifecycle beyond the development teams traditionally associated with agile and include system engineering, testing, validation and verification, certification and accreditation. We have observed the ramifications of this misalignment during the integration of components built by different teams, where incompatibilities lead to significant rework.

To map, analyze, and support the architectural decisions of industry collaborators, we plan to map our MDM approach into a conceptual model developed by Kevin Sullivan of the University of Virginia, who is working on creating a cyber-social conceptual model. Sullivan's work focuses on the social networks and the value of relationships between decision makers in a system.

Challenges

A primary technical challenge that we face in this approach involves scaling. As a practice, Agile has been successful in helping to solidify the efficiency of development teams when projects involve small teams. Applying these same concepts to large-scale distributed systems-- including the rest of the organization that has priorities larger than the development team--will be critical for success, but it will also present some of the greatest challenges. To address the challenge of scaling, we are looking at the influence architecture exerts on managing teams and how to provide practical guidance on what amount of architecture is needed and when. On the one hand, early or overproduction of architecture can create delay. On the other hand, not enough production of architecture can result in integration defects leading to rework. The focus of lean thinking on improving cycle time by eliminating waste, in the form of delay or unnecessary rework, in conjunction with architecture, has shown great potential for improving management of software development projects and increasing flow of value delivered to the customer.

Next Steps

We are in the process of conducting a survey of critical agile development properties and will write about the results in a future blog posting.

Additional Resources:

To read the SEI technical report, Agile Methods: Selected DoD Management and Acquisition Concerns, please visit
www.sei.cmu.edu/library/abstracts/reports/11tn002.cfm