Major acquisition programs increasingly rely on software to provide substantial portions of system capabilities. All too often, however, software is not considered when the early, most constraining program decisions are made. SEI researchers have identified misalignments between software architecture and system acquisition strategies that lead to program restarts, cancellations, and failures to meet important missions or business goals. This blog posting, the second installment in a two-part series, builds on the discussions in part one by introducing several patterns of misalignment (i.e., anti-patterns) that we've identified in our research and discussing how these anti-patterns are helping us create a new method for aligning software architecture and system acquisition strategies to reduce project failure.

Identifying Anti-Patterns

We used an interview-based approach to discover and document patterns of alignment among four key aspects: business and mission goals, architecture, quality attributes, and acquisition strategy. We then analyzed the interview data looking for evidence of alignment or misalignment. Since most of our data comes from troubled programs, we have primarily discovered evidence of misalignment--known as anti-patterns--to date.
Our initial set of anti-patterns includes

• undocumented business goals - the lack of well-documented business goals expressed as they apply to an acquisition program
• unresolved conflicting goals - the lack of analysis and reconciliation of known goals
• failure to adapt - failure of an acquisition program to modify the architecture and the acquisition strategy in response to changing goals, priorities, or technology
• turbulent acquisition environment - requested changes are so frequent and contradictory that an acquisition program cannot realistically accommodate them
• poor consideration of software - critical decisions made early in an acquisition program's lifecycle have strong negative implications on the system's software
• inappropriate acquisition strategies - the acquisition strategy fails to consider important software attributes
• overlooking quality attributes - a failure to define and use software quality attributes in the definition of the software architecture or acquisition strategy

Let's explore one of these anti-patterns to show how it might be used. The first anti-pattern--undocumented business goals--reflects a lack of precise, well-defined, and well-documented business goals for a DoD acquisition program. In the programs we examined, we found that these goals were seldom explicitly expressed (e.g., "replace legacy system") or they reflected high-level program constraints (e.g., "maximize competition") or policy regulations (e.g., "implement an open architecture").

When this anti-pattern is present, we found that mission requirements dominate the definition of the software architecture, often leading to an architecture contrary to the achievement of the unstated business goal. For instance, an architect might reasonably design a monolithic architecture that was an excellent fit for the mission goals for performance but which could be strongly at odds with an implicit but unspecified business goal to avoid vendor lock.

The lack of explicit business goals has a more direct impact on the acquisition strategy. In one program in our study, a key element for the program was to build a new system with significant new capabilities. The acquisition strategy specified a slow, deliberate pace to ensure that the new capability was defined correctly. A competing goal was to replace several "end-of-life" systems. Not stated in this goal was the urgent need to replace these failing system as quickly as possible. When the operators and maintainers of the legacy systems became aware of the intended acquisition strategy, they forced a major change in program focus. The sequence of acquisition activities required alteration, which caused a significant delay in meeting either goal.

Creating a Method for Identifying Misalignments

Discovering and documenting anti-patterns is only the beginning of our work in addressing the problems of misalignment. The second phase of our project involves creating a method that helps acquisition programs avoid the anti-patterns we've discovered and provides options that could help programs better align their acquisition strategy and software architecture to satisfy stakeholder mission and business goals. We will then validate the utility of this method through the help of projects and programs outside the SEI.

Software-reliant systems are inherently social and technical endeavors. A key facet of our method, therefore, is thus its ability to bring disparate actors together--often for the first time--to rationally identify and discuss issues of mutual concern and be able to make hard and informed choices based on rational information.
We plan to adapt and tailor existing methods where possible. We are currently exploring methods in the following areas:

• identifying salient stakeholders - There are many requirements elicitation and analysis methods. Unfortunately, most methods assume that it is possible to know which stakeholders will most affect or be most affected by the program. We are therefore considering Controlled Requirements Expression (CORE), which is a method that assists developers in identifying stakeholders related to a given acquisition to help develop a more complete list of stakeholders.
• defining business and mission goals - The Pedigreed Attribute eLicitation Method (PALM) discussed in part one of this blog series remains a central element of our new method. PALM enables organizations to systematically identify the high-priority mission and business goals from system stakeholders. The architectural implications of those goals are then captured by determining the quality attribute requirements they imply. We are extending PALM to investigate the acquisition strategy implications of business or mission goals.
• analyzing quality attributes - Quality Attribute Workshops (QAW) are a widely accepted method for developing definitions of the quality attributes that form the basis for deriving the software architecture. We are using the same approach to derive attributes that should drive the acquisition strategy.
• trading off architecture and acquisition strategy options - analysis methods, such as Architecture Tradeoff Analysis Method (ATAM) or Cost Benefit Analysis Method (CBAM), are used to ensure consistency of software and system quality attributes. We are analyzing these methods to explore consistency between the acquisition strategy and its driving quality attributes.

Looking Ahead

Government acquisitions are more likely to succeed if a program can align its acquisition strategy and software architecture with each other and with respect to satisfying stakeholder mission and business goals. Our research has shown evidence of misalignments in the form of anti-patterns. Discovering the patterns a program should avoid is a key step toward our objective to develop a method that systematically supports business and mission goals by aligning the acquisition strategy and software architecture.

We welcome opportunities to validate and expand the anti-patterns or pilot our emerging method. Please leave us feedback or questions about our research in the comments section below and we will follow up with you.

Additional Resources

Read the first post in this series Reducing Project Failures by Aligning Acquisition Strategy and Software Architecture with Stakeholder Needs.

For more information about the Pedigreed Attribute eLicitation Method (PALM), please visit
https://resources.sei.cmu.edu/library/asset-view.cfm?assetid=9347.

To read the SEI technical report, Quality Attribute Workshops, please visit
https://resources.sei.cmu.edu/library/asset-view.cfm?assetID=6687

For more information about the book, Evaluating Software Architectures: Methods and Case Studies, please visit www.sei.cmu.edu/library/abstracts/books/020170482X.cfm

To read the SEI technical report, Making Architecture Design Decisions: An Economic Approach, please visit
www.sei.cmu.edu/library/abstracts/reports/02tr035.cfm