All software engineering and management practices are based on cultural and social assumptions. When adopting new practices, leaders often find mismatches between those assumptions and the realities within their organizations. The SEI has an analysis method called Readiness and Fit Analysis (RFA) that allows the profiling of a set of practices to understand their cultural assumptions and then to use the profile to support an organization in understanding its fit with the practices' cultural assumptions. RFA has been used for multiple technologies and sets of practices, most notably for adoption of CMMI practices.
One of the fundamental principles of technology adoption is that of mutual adaptation. This principle asserts that a successful technology adoption by an organization usually requires adaptation of both the technology and the organization. The technology may adapt, for example, by being configurable - allowing switching on or off of different features - or by allowing localization to a different native language. The organization may adapt by changing some of its business workflows so they are more compatible with the technology or by changing the roles of the people involved in different processes that are affected by the technology.
When an organization adopts a new set of practices, it sees many of the same issues associated with adopting a new hardware or software technology. The SEI has observed that when adopting new practices--as when adopting new technologies--the principle of mutual adaptation applies. One of our observations has been that the closer the organization's culture is to the implied cultural assumptions of a set of practices, the easier it is for that organization to adopt those practices.
As part of our research in the adoption of agile methods in U.S. DoD settings, we have adapted the RFA profiling technique to accommodate both the typical factors used in RFA and some factors that are more uniquely associated with the DoD acquisition environment. We found that only applying the commercial profile didn't highlight enough of the issues that we were seeing in our interviews and observations of practice. A technical note on RFA factors for agile adoption in DoD will be published at a future date.
In this post, we want to present the categories and factors that we have identified so far, with the help of our interviewees and our SEI Agile Collaboration Group. This latter group consists of over a dozen DoD and other federal government acquisition practitioners, plus several DoD contractor organization representatives who are all actively adopting various relevant Agile methods in their organizations. We have characterized the following six categories to profile for readiness and fit:
business and acquisition - adoption factors related to business strategy, acquisition strategy, and contracting mechanisms
organizational climate - adoption factors related to sponsorship, leadership, reward systems, values, and similar "soft" issues
system attributes - adoption factors related to the actual characteristics of the system(s) being developed
project and customer environment - adoption factors related to project management norms, team dynamics and support structures, and customer relationships and expectations
technology environment - adoption factors related to the technologies that are in place or planned to support the selected agile methods
practices - a taxonomy of agile practices that is used to understand which practices an organization plans to adopt so that other factors can be calibrated around those expectations
If an organization has used RFA in other settings, the factors that were found in the original RFA are scattered among the business and acquisition, organizational climate, and project and customer environment categories.
Each category has a set of attributes that can be characterized by a statement that would represent what you would expect to see if you were observing a successful agile project or organization operating in relation to that attribute. For example, an attribute of business and acquisition is stated as:
Oversight mechanisms are aligned with agile principles.
Oversight is an aspect of acquisition that can either support or disable an agile project. Alignment of oversight with agile principles thus reduces the risk that oversight will be counterproductive.
The remainder of this blog posting describes key factors in the business and acquisition category. In future posts, we will explore other categories of factors that deal with issues that cause different challenges to adoption of agile mthods.
The Business and Acquisition Category
This category covers issues related to an organization's business strategy or mission and some specific factors related to acquisition and contracting. Business strategy is an important fit element because many organization values and principles are tied to the strategy. If the strategy changes, the organization's values may change, creating either a better or worse fit environment for a particular set of practices. Similarly, in DoD settings, certain contracting approaches are more aligned with particular sets of values and practices, and changing the way a contract is formulated can have a significant impact on the values and practices that will be needed to execute that contract. The following list has both a short title that summarizes the statement and a statement that provides a condition or behavior found in an organization successfully using engineering and management methods consistent with agile principles as published in the Agile Manifesto.
Clear Program Goals. Business or program goals are clear and reflect stakeholder concerns. From an agile methods perspective, the organization's mission or business goals are one of the touchpoints for decision making. If they are not clear--or if they do not adequately reflect the concerns of the organization's stakeholders--then lower level decision-making runs the risk of being misaligned with the organization's focus.
Defined Success Strategies. Success strategies (e.g., roadmaps, product portfolios, etc.) are defined and clearly communicated.
From an agile methods perspective, being clear about the roadmaps, portfolios, etc. that an organization uses to define its productivity and successful completions is a key to understanding how an individual project fits into the broader organizational mission.
Project Funding Secured. Funding for the project has been secured.
This factor may seem obvious and one that is a success criterion for any project, which is true. Of particular importance when applying agile methods to DoD organizations, however, is that there are multiple ways to fund and contract for information technology products and services. Some steps in the formulation of a program can be executed prior to official funding, but there are many tasks that cannot be initiated until the funding allocation process has completed.
Close Stakeholder/Developer Collaboration Enabled. Mechanisms are in place in the contract and acquisition strategy to allow close collaboration between developers and other stakeholders (e.g., certification and accreditation personnel, end users, and others). The fourth principle derived from the Agile Manifesto states, "Business people and developers must work together daily throughout the project." In a commercial environment, business people includes managers of the project and end users of the product being developed. In the DoD, these roles may be in different organizations, and there are multiple business-related stakeholder roles to account for--program office personnel, information assurance, independent verification and validation agents, end users, logisticians, trainers, and others. If the acquisition strategy and associated contract vehicles create barriers to collaboration among these roles and the developer, it will be hard to achieve the performance of shoulder-to-shoulder agile implementations.
Interim Delivery Enabled. Mechanisms are in place in the contract and acquisition strategy that allow for interim demonstration and delivery between official releases.
The first principle derived from the Agile Manifesto states, "Our highest priority is to satisfy the customer through early and continuous delivery of valuable software." DoD contracts can specify the cadence of delivery in the Statement of Work (SOW) and also in the way they apply different standards and define line items in their Contract Data Requirements List. If a contract specifies a single delivery of the software, other mechanisms may be in place to prevent productive early demonstration and re-orienting of priorities or focus.
Oversight Supports Agile Principles. Contract oversight mechanisms are aligned with agile principles.
As with delivery enablement, the contract is the mechanism wherein program office technical and management oversight is specified. Contracts for large acquisition programs typically mandate document-centric capstone reviews, such as Preliminary Design Reviews (PDRs) and Critical Design Reviews (CDRs). These reviews analyze requirement, preliminary design (PDR), and detailed design (CDR) documentation; software development does not begin until after all these documents have been approved following the CDR. This linear lifecycle model is not as productive an oversight strategy for contracts employing agile methods, where contracting language enables incremental, more frequent (and less formal) progress reviews. Beyond the contract language itself, the expectations of reviewers and oversight personnel must also be set appropriately.
Clear Alignment of Software Goals/Program Goals. The alignment of software-related goals with program-level goals is clear.
This factor is also important in non-agile settings, but its urgency in agile settings comes from the fact that software will be available earlier to test and interact with the other parts of the system. For systems engineers unaccustomed to this early access, provisioning test beds consisting of hardware emulators and simulation environments may not get the attention needed to ensure the software part of the program can take advantage of incremental deliveries.
Appropriate Contract Type. Contract type accounts for use of agile or lean methods in the program.
This factor may seem obvious, but it's actually quite a challenge for DoD program offices. Almost any contract type (firm fixed price, indefinite deliver/indefinite quality, time and materials, level of effort, cost plus incentive fee, etc.) can be used to effectively support development using agile methods. For each contract type, however, the way the agreement is framed determines how effective it will be. The contract type and the acquisition strategy must therefore be aligned to support agile methods implementation.
Appropriate Lifecycle Activities. Lifecycle activities that are planned in the acquisition strategy are compatible with agile methods.
It's not enough that the contract vehicle be written correctly. It's also important that the life cycle activities are specified in a way that can leverage the iterative and incremental nature of agile software development. For example, building test support equipment and test suites early in the life cycle is essential if test-driven development is an agile method being applied.
Agile at-Scale Enabled. The acquisition strategy takes into account the use of agile methods at the scale needed for the program.
The most prevalent use to date for agile methods has been on smaller projects, but even in the DoD there have been successful projects with dozens of developers. To appropriately express the agile principles, stakeholders must consider communication mechanisms, architectural patterns, and layered management approaches. If these factors are not taken into account in the acquisition strategy, larger agile implementations may not be resourced effectively.
Upcoming blog entries in this series will describe factors in organizational climate, system attributes, and technology environment. We will also describe factors in the project and customer environment and practices categories that form a picture of the organization planning to adopt agile practices to help them understand where they are likely to have challenges adopting the desired practices. From there, risk mitigation and issue management strategies can be defined to minimize the probability and/or impact of the adoption risks that have been identified.
We welcome feedback and comments on both the concept and the content of the model so far. We solicit your feedback, especially if you are a practitioner that is being asked to adapt your own work to accommodate agile methods.
Object-oriented programs present considerable challenges to reverse engineers. For example, C++ classes are high-level structures that lead to complex arrangements of assembly instructions when compiled. These complexities are exacerbated for malware analysts because malware rarely has source code available; thus, analysts must grapple with sophisticated data structures exclusively at the machine code level. As more and more object-oriented malware is written in C++, analysts are increasingly faced with the challenges of reverse engineering C++ data structures. This blog post is the first in a series that discusses tools developed by the Software Engineering Institute's CERT Division to support reverse engineering and malware analysis tasks on object-oriented C++ programs.