It is widely recognized that the Department of Defense (DoD) needs to have a nimble response to nimble adversaries. However, the inflexibility of many DoD development and acquisition practices begets inflexible architectures that often slow progress and increase risk to operational forces. This rejection of modern development methods actually increases program risk and extends development timelines, effectively reducing the value of the DoD's acquisition portfolio. As a result, the current lack of capacity for breadth and pace of change impedes our ability to evolve capability quickly and robustly enough to meet new requirements in emerging technical and warfighting environments. The push to deliver artificial intelligence (AI)-based capabilities (and respond effectively to AI-based threats) increases both the urgency and importance of addressing the challenges.
This post is also co-authored by Douglas C. Schmidt and William Scherlis.
In its effort to increase the capability of the warfighter, the Department of Defense (DoD) has made incremental changes in its acquisition practices for building and deploying military capacity. This capacity can be viewed as "platforms" (tanks, ships, aircraft, etc.) and the mission system "payloads" (sensors, command and control, weapons, etc.) that are populated onto those platforms to deliver the desired capability. This blog post, the first in a series excerpted from a recently published paper, explores opportunities in modularity and open systems architectures with the aim of helping the DoD deliver higher quality software to the warfighter with far greater innovation in less time.
The mix of program-scale Agile and technical baseline ownership drives cheaper, better, and faster deployment of software-intensive systems. Although these practices aren't new, the SEI has seen how their combination can have dramatic effects. The Air Force Distributed Common Ground System (AF DCGS)--the Air Force's primary weapon system for intelligence, surveillance, reconnaissance, planning, direction, collection, processing, exploitation, analysis, and dissemination--employs a global communications architecture that connects multiple intelligence platforms and sensors. The AF DCGS challenge is to bring new sensors and processing applications online quickly and efficiently. Other large government software-intensive systems face similar challenges. The SEI has found that Agile cultural transformation--along with strong technical baseline ownership--is critical for programs like DCGS to deliver new capability faster and with greater confidence. These strategies working together can help create incremental and iterative approaches to deliver more frequent and more manageable technical capability. In this blog, I present the SEI's experiences in helping large Department of Defense (DoD) programs, such as AF DCGS, use concepts like owning the technical baseline and Agile software development techniques to deliver new capability on a regular basis.
This is the first post in a three-part series.
Software and acquisition professionals often have questions about recommended practices related to modern software development methods, techniques, and tools, such as how to apply agile methods in government acquisition frameworks, systematic verification and validation of safety-critical systems, and operational risk management. In the Department of Defense (DoD), these techniques are just a few of the options available to face the myriad challenges in producing large, secure software-reliant systems on schedule and within budget.
In an effort to offer our assessment of recommended techniques in these areas, SEI built researchers built upon an existing collaborative online environment known as SPRUCE (Systems and Software Producibility Collaboration Environment), hosted on the Cyber Security & Information Systems Information Analysis Center (CSIAC) website. From June 2013 to June 2014, the SEI assembled guidance on a variety of topics based on relevance, maturity of the practices described, and the timeliness with respect to current events. For example, shortly after the Target security breach of late 2013, we selected Managing Operational Resilience as a topic.
Ultimately, SEI curated recommended practices on five software topics: Agile at Scale, Safety-Critical Systems, Monitoring Software-Intensive System Acquisition Programs, Managing Intellectual Property in the Acquisition of Software-Intensive Systems, and Managing Operational Resilience. In addition to a recently published paper on SEI efforts and individual posts on the SPRUCE site, these recommended practices will be published in a series of posts on the SEI blog. This post, the first in a three-part series by Robert Ferguson, first explores the challenges to Monitoring Software-Intensive System Acquisition (SISA) programs and presents the first two recommended best practices as detailed in the SPRUCE post. The second post in this series will present the next three best practices. The final post will present the final two recommendations as well as conditions that will allow organizations to derive the most benefit from these practices.
Acquisition executives in domains ranging from modernizing legacy business systems to developing real-time communications systems often face the following challenge:
Vendors claim that model-driven engineering (MDE) tools enable developers to generate software code automatically and achieve extremely high developer productivity.
More and more, suppliers of software-reliant Department of Defense (DoD) systems are moving away from traditional waterfall development practices in favor of agile methods. As described in previous posts on this blog, agile methods are effective for shortening delivery cycles and managing costs. If the benefits of agile are to be realized effectively for the DoD, however, personnel responsible for overseeing software acquisitions must be fluent in metrics used to monitor these programs. This blog post highlights the results of an effort by researchers at the Carnegie Mellon University Software Engineering Institute to create a reference for personnel who oversee software development acquisition for major systems built by developers applying agile methods. This post also presents seven categories for tracking agile metrics.
As part of an ongoing effort to keep you informed about our latest work, I would like to let you know about some recently published SEI technical reports and notes. These reports highlight the latest work of SEI technologists in cybersecurity risks, software assurance, advanced persistent threat, international insider threat, Wireless Emergency Alerts Service, security and survivability, and acquisition.
Although software is increasingly important to the success of government programs, there is often little consideration given to its impact on early key program decisions. The Carnegie Mellon University Software Engineering Institute (SEI) is conducting a multi-phase research initiative aimed at answering the question: is the probability of a program's success improved through deliberately producing a program acquisition strategy and software architecture that are mutually constrained and aligned?
The Department of Defense (DoD) has become deeply and fundamentally reliant on software. As a federally funded research and development center (FFRDC), the SEI is chartered to work with the DoD to meet the challenges of designing, producing, assuring, and evolving software-reliant systems in an affordable and dependable manner. This blog post--the first in a multi-part series--outlines key elements of the forthcoming SEI Strategic Research Plan that addresses these challenges through research and acquisition support and collaboration with DoD, other federal agencies, industry, and academia.
Organizational improvement efforts should be driven by business needs, not by the content of improvement models. While improvement models, such as the Capability Maturity Model Integration (CMMI) or the Baldrige Criteria for Performance Excellence, provide excellent guidance and best practice standards, the way in which those models are implemented must be guided by the same drivers that influence any other business decision. Business drivers are the collection of people, information, and conditions that initiate and support activities that help an organization accomplish its mission.
Engineering the architecture for a large and complex system is a hard, lengthy, and complex undertaking. System architects must perform many tasks and use many techniques if they are to create a sufficient set of architectural models and related documents that are complete, consistent, correct, unambiguous, verifiable, usable, and useful to the architecture's many stakeholders. This blog posting, the second in a two-part series, takes a deeper dive into the Method Framework for Engineering System Architectures (MFESA), which is a situational process engineeringframework for developing system-specific methods to engineer system architectures.
As part of an ongoing effort to keep you informed about our latest work, I'd like to let you know about some recently published SEI technical reports and notes. These reports highlight the latest work of SEI technologists in workforce competency and readiness, cyber forensics, exploratory research, acquisition, and software-reliant systems. This post includes a listing of each report, author(s), and links where the published reports can be accessed on the SEI website.
Major acquisition programs increasingly rely on software to provide substantial portions of system capabilities. All too often, however, software is not considered when the early, most constraining program decisions are made. SEI researchers have identified misalignments between software architecture and system acquisition strategies that lead to program restarts, cancellations, and failures to meet important missions or business goals. This blog posting--the second installment in a two-part series--builds on the discussions in part one by introducing several patterns of misalignment--known as anti-patterns--that we've identified in our research and discussing how these anti-patternsare helping us create a new method for aligning software architecture and system acquisition strategies to reduce project failure.
Engineering the architecture for a large and complex system is a hard, lengthy, and complex undertaking. System architects must perform many tasks and use many techniques if they are to create a sufficient set of architectural models and related documents that are complete, consistent, correct, unambiguous, verifiable, and both usable by and useful to the architecture's many stakeholders. This blog posting, the first in a two-part series, presents the Method Framework for Engineering System Architectures (MFESA), which is a situational process engineeringframework for developing system-specific methods to engineer system architectures. This posting provides a brief historical description of situational method engineering, explains why no single system architectural engineering method is adequate, and introduces MFESA by providing a top-level overview of its components, describing its applicability, and explaining how it simultaneously provides the benefits of standardization and flexibility.
Major acquisition programs increasingly rely on software to provide substantial portions of system capabilities. Not surprisingly, therefore, software issues are driving system cost and schedule overruns. All too often, however, software is not even a consideration when the early, most constraining program decisions are made. Through analysis of troubled programs, SEI researchers have identified misalignments between software architecture and system acquisition strategies that lead to program restarts, cancellations, and failures to meet important missions or business goals. To address these misalignments, the SEI is conducting new research on enabling organizations to reduce program failures by harmonizing their acquisition strategy with their software architecture.
The extent of software in Department of Defense (DoD) systems has increased by more than an order of magnitude every decade. This is not just because there are more systems with more software; a similar growth pattern has been exhibited within individual, long-lived military systems. In recognition of this growing software role, the Director of Defense Research and Engineering (DDR&E, now ASD(R&E)) requested the National Research Council (NRC) to undertake a study of defense software producibility, with the purpose of identifying the principal challenges and developing recommendations regarding both improvement to practice and priorities for research.
New acquisition guidelines from the Department of Defense (DoD) aimed at reducing system lifecycle time and effort are encouraging the adoption of Agile methods. There is a general lack, however, of practical guidance on how to employ Agile methods effectively for DoD acquisition programs. This blog posting describes our research on providing software and systems architects with a decision making framework for reducing integration risk with Agile methods, thereby reducing the time and resources needed for related work.
In my preceding blog post, I promised to provide more examples highlighting the importance of software sustainmentin the US Department of Defense (DoD). My focus is on certain configurations of weapons systems that are no longer in production for the United States Air Force, but are expected to remain a key component of our defense capability for decades to come, and thus software upgrade cycles need to refresh capabilities every 18 to 24 months. Throughout this series on efficient and effective software sustainment, I will highlight examples from each branch of the military. This second blog post describes effective sustainment engineering efforts in the Air Force, using examples from across the service's Air Logistics Centers (ALCs).
Our SEI blog has included thoughtful discussions about sustaining software, such as the two-part post "The Growing Importance of Sustaining Software for the DoD." Software sustainment is growing in importance as the lifetimes of hardware systems greatly exceed the normal lifetime of software systems they are partnered with, as well as when system functionality increasingly depends on software elements. This blog post--the first in a multi-part series--provides specific examples of the importance of software sustainment in the Department of Defense (DoD), where software upgrade cycles need to refresh capabilities every 18 to 24 months on weapon systems that have been out of production for many years, but are expected to maintain defense capability for decades.
We use the SEI Blog to inform you about the latest work at the SEI, so this week I'm summarizing some video presentations recently posted to the SEI website from the SEI Technologies Forum. This virtual event held in late 2011 brought together participants from more than 50 countries to engage with SEI researchers on a sample of our latest work, including cloud computing, insider threat, Agile development, software architecture, security, measurement, process improvement, and acquisition dynamics. This post includes a description of all the video presentations from the first event, along with links where you can view the full presentations on the SEI website.
Over the past several years, the SEI has explored the use of Agile methods in DoD environments, focusing on both if and when they are suitable and how to use them most effectively when they are suitable. Our research has approached the topic of Agile methods both from an acquisition and a technical perspective. Stephany Bellomo described some of our experiences in previous blog posts What is Agile? and Building a Foundation for Agile. This post summarizes a project the SEI has undertaken to review and study Agile approaches, with the goal of developing guidance for their effective application in DoD environments.
In our work with acquisition programs, we've often observed a major problem: requirements specifications that are incomplete, with many functional requirements missing. Whereas requirements specifications typically specify normal system behavior, they are often woefully incomplete when it comes to off-nominal behavior, which deals with abnormal events and situations the system must detect and how the system must react when it detects that these events have occurred or situations exist. Thus, although requirements typically specify how the system must behave under normal conditions, they often do not adequately specify how the system must behave if it cannot or should not behave as normally expected. This blog post examines requirements engineering for off-nominal behavior.
A key mission of the SEI is to advance the practice of software engineering and cyber security through research and technology transition to ensure the development and operation of software-reliant Department of Defense (DoD) systems with predictable and improved quality, schedule, and cost. To achieve this mission, the SEI conducts research and development (R&D) activities involving the DoD, federal agencies, industry, and academia. One of my initial blog postings summarized the new and upcoming R&D activitieswe had planned for 2011. Now that the year is nearly over, this blog posting presents some of the many R&D accomplishments we completed in 2011.
As part of an ongoing effort to keep you informed about our latest work, I'd like to let you know about some recently published SEI technical reports and notes. These reports highlight the latest work of SEI technologists in Agile methods, insider threat,the SMART Grid Maturity Model, acquisition, and CMMI. This post includes a listing of each report, author/s, and links where the published reports can be accessed on the SEI website.
Background: In our research and acquisition work on commercial and Department of Defense (DoD) programs, we see many systems with critical safety and security ramifications. With such systems, safety and security engineering are used to managing the risks of accidents and attacks. Safety and security requirements should therefore be engineered to ensure that residual safety and security risks will be acceptable to system stakeholders. The first post in this series explored problems with quality requirements in general and safety and security requirements in particular. The second post, took a deeper dive into key obstacles that acquisition and development organizations encounter concerning safety- and security-related requirements. This post introduces a collaborative method for engineering these requirements that overcomes the obstacles identified in earlier posts.
Happy Labor Day from all of us here at the SEI. I'd like to take advantage of this special occasion to keep you apprised of some recent technical reports and notes from the SEI. It's part of an ongoing effort to keep you informed about our latest work. These reports highlight the latest work of SEI technologists in architecting service-oriented systems, operational resilience, standards-based automated remediation, and acquisition. This post includes a listing of each report, author/s, and links where the published reports can be accessed on the SEI website.
Background: Over the past decade, the U.S. Air Force has asked the SEI's Acquisition Support Program (ASP) to conduct a number of Independent Technical Assessments (ITAs) on acquisition programs related to the development of IT systems; communications, command and control; avionics; and electronic warfare systems. This blog posting is the latest installment in a series that explores common themes across acquisition programs that we identified as a result of our ITA work. Previous themes explored in this series include Misaligned Incentives, The Need to Sell the Program, and The Evolution of "Science Projects."This post explores the fourth theme: common infrastructure and joint programs, which describes a key issue that arises when multiple organizations attempt to cooperate in the development of a single system, infrastructure, or capability that will be used and shared by all parties.
Background: In our research and acquisition work on commercial and Department of Defense (DoD) programs, ranging from relatively simple two-tier data-processing applications to large-scale multi-tier weapons systems, one of the primary problems that we see repeatedly is that acquisitionand development organizations encounter the following three obstacles concerning safety- and security-related requirements:
Background: Over the past decade, the U.S. Air Force has asked the SEI's Acquisition Support Program (ASP) to conduct a number of Independent Technical Assessments (ITAs) on acquisition programs related to the development of IT systems, communications, command and control, avionics, and electronic warfare systems. This blog post is the third in a series that enumerates common themes across acquisition programs that we identified as a result of our ITA work. Other themes explored in this series include misaligned incentives, the need to sell the program, and common infrastructure and joint programs. This post explores the third theme in this series, the evolution of "science projects," which describes how prototype projects that unexpectedly grow in size and scope during development often have difficulty transitioning into a formal acquisition program.
Background: The U.S. Air Force has sponsored a number of SEI Independent Technical Assessments (ITAs) on acquisition programs that operated between 2006 and 2009. The programs focused on the development of IT systems, communications, command and control, avionics, and electronic warfare systems. This blog post is the second in a series that identifies four themes across acquisition programs that the SEI identified as a result of our ITA work. Other themes explored in the series include misaligned incentives, the evolution of science projects, and common infrastructure and joint programs. This post explores a related second theme, the need to sell the program, which describes a situation in which people involved with acquisition programs have strong incentives to "sell" those programs to their management, sponsors, and other stakeholders so that they can obtain funding, get them off the ground, and keep them sold.
Background:Over the past decade, the U.S. Air Force has asked the SEI's Acquisition Support Program (ASP) to conduct a number of Independent Technical Assessments (ITAs) on acquisition programs related to the development of IT systems; communications, command and control; avionics; and electronic warfare systems. This blog post is the first in a series that identifies common themes across acquisition programs that we identified as a result of our ITA work. This post explores the first theme: misaligned incentives, which occur when different individuals, groups, or divisions are rewarded for behaviors that conflict with a common organizational goal.
This is a second in a series of posts focusing on Agile software development. In the first post, "What is Agile?" we provided a short overview of the key elements of the Agile approach, and we introduced the Agile Manifesto. One of the guiding principles from the manifesto emphasizes valuing people over developing processes. While the manifesto clearly alludes to the fact that too much focus on process (and not results) can be a bad thing, we introduce the notion here that the other end of the spectrum can also be bad. This blog explores the level of skill that is needed to develop software using Agile (do you need less skill or more?), as well as the importance of maintaining strong competency in a core set of software engineering processes.
If you ask the question, "What is Agile?" you are likely to get lots of different answers. That's because there is no universally accepted formal definition for Agile. To make matters worse, there are ongoing debates over what Agile software development SHOULD mean. That being the case, when answering the question, "What is Agile?" the safest bet is to stick to what people can agree on, and people generally agree on three key elements of Agile. Taken together, these describe the Agile software development method, as well as the software development approach. In this post--the first in a series on Agile--I will explain the foundations of Agile and its use by developers.
Large-scale DoD acquisition programs are increasingly being developed atop reusable software platforms--known as Common Operating Environments (COEs) --that provide applications and end-users with many net-centric capabilities, such as cloud computing or Web 2.0 applications, including data-sharing, interoperability, user-centered design, and collaboration. Selecting an appropriate COE is critical to the success of acquisition programs, yet the processes and methods for evaluating COEs had not been clearly defined. I explain below how the SEI developed a Software Evaluation Framework and applied it to help assess the suitability of COEs for the US Army.
Strategic planning is a process for defining an organization's approach for achieving its mission. Conducting successful strategic planning is essential because it creates a foundation for executing work, as well as setting the stage for enterprise architecture, process improvement, risk management, portfolio management, and any other enterprise-wide initiatives. Government organizations are operating in an environment of almost near-constant change, however, which makes it hard to conduct strategic planning efforts successfully. Moreover, when organizations do tackle strategy, they often get bogged down reflecting on the past or speculating on an uncertain future. This posting describes recent work investigating and integrating techniques to improve organizational strategic planning.