There's been a widespread movement in recent years from traditional waterfall development to Agile approaches in government software acquisition programs. This transition has created the need for personnel who oversee government software acquisitions to become fluent in metrics used to monitor systems developed with Agile methods. This post, which is a follow-up to my earlier post on Agile metrics, presents updates on our Agile-metrics work based on recent interactions with government programs.
This post is co-authored by Will Hayes and Eileen Wrubel.
On July 14, 2016, the House Ways and Means Subcommittee on Social Security convened a hearing on the Social Security Administration's (SSA) information technology modernization plan. The hearing focused on the current state of the Social Security Administration's (SSA) Information Technology (IT) modernization plan and best practices for IT modernization, including oversight of agile software development. Agile development approaches, relatively new in government settings, create opportunities for rapid deployment of new capabilities but also pose challenges to traditional government oversight and management practices. A team of researchers from the SEI's Agile in Government (AIG) team were part of an expert panel brought in to testify before the members of the subcommittee. The team, comprised of me, AIG principal engineer Will Hayes, and AIG program manager Eileen Wrubel, developed written testimony that was submitted to the committee in conjunction with verbal testimony delivered by Hayes. This blog post, the first in a series, presents the written testimony as submitted to Congress, drawing upon seven years of research the SEI has conducted on the use of Agile in government settings. Specifically, this post provides a summary of challenges observed by the SEI in overseeing Agile programs in governmentsuch as progress measurements, IT transformations beyond Agile, and workforce development of government staff working in Agile settings.
The mix of program-scale Agile and technical baseline ownership drives cheaper, better, and faster deployment of software-intensive systems. Although these practices aren't new, the SEI has seen how their combination can have dramatic effects. The Air Force Distributed Common Ground System (AF DCGS)--the Air Force's primary weapon system for intelligence, surveillance, reconnaissance, planning, direction, collection, processing, exploitation, analysis, and dissemination--employs a global communications architecture that connects multiple intelligence platforms and sensors. The AF DCGS challenge is to bring new sensors and processing applications online quickly and efficiently. Other large government software-intensive systems face similar challenges. The SEI has found that Agile cultural transformation--along with strong technical baseline ownership--is critical for programs like DCGS to deliver new capability faster and with greater confidence. These strategies working together can help create incremental and iterative approaches to deliver more frequent and more manageable technical capability. In this blog, I present the SEI's experiences in helping large Department of Defense (DoD) programs, such as AF DCGS, use concepts like owning the technical baseline and Agile software development techniques to deliver new capability on a regular basis.
Organizations and federal agencies seeking to adopt Agile often struggle because they do not understand the adoption risks involved when contemplating the use of Agile approaches. This ongoing series on Readiness and Fit Analysis (RFA) focuses on helping federal agencies, such as the Department of Defense, the Internal Revenue Service, the Food and Drug Administration, and other organizations in regulated settings, understand the risks involved when contemplating or embarking on a new approach to developing or acquiring software. This blog post, the seventh in a series, explores issues related to the technology environment that organizations should consider when adopting Agile approaches.
According to the National Institute of Standards and Technology (NIST), Information Security Continuous Monitoring (ISCM) is a process for continuously analyzing, reporting, and responding to risks to operational resilience (in an automated manner, whenever possible). Compared to the traditional method of collecting and assessing risks at longer intervals--for instance, monthly or annually--ISCM promises to provide near-real-time situational awareness of an organization's risk profile. ISCM creates challenges as well as benefits, however, because the velocity of information gathered using ISCM is drastically increased. Development, operation, and maintenance processes built for a more leisurely pace can thus be overwhelmed. This blog post explores how organizations can leverage Agile methods to keep pace with the increased velocity of ISCM risk information, while ensuring that changes to systems are conducted in a controlled manner.
1. Team coordination 2. Architectural runway 3. Align development and decomposition. 4. Quality-attribute scenarios 5. Test-driven development
This post presents the remaining five technical best practices, as well as three conditions that will help organizations achieve the most value from these recommended practices. This post was originally published in its entirety on the SPRUCE website.
This post is the first in a two-part series highlighting 10 recommended practices for achieving agile at scale.
Software and acquisition professionals often have questions about recommended practices related to modern software development methods, techniques, and tools, such as how to apply agile methods in government acquisition frameworks, systematic verification and validation of safety-critical systems, and operational risk management. In the Department of Defense (DoD), these techniques are just a few of the options available to face the myriad challenges in producing large, secure software-reliant systems on schedule and within budget.
The SEI Blog continues to attract an ever-increasing number of readers interested in learning more about our work in agile metrics, high-performance computing, malware analysis, testing, and other topics. As we reach the mid-year point, this blog posting highlights our 10 most popular posts, and links to additional related resources you might find of interest (Many of our posts cover related research areas, so we grouped them together for ease of reference.)
Before we take a deeper dive into the posts, let's take a look at the top 10 posts (ordered by number of visits, with #1 being the highest number of visits):
In 2010, the Office of Management and Budget (OMB) issued a 25-point plan to reform IT that called on federal agencies to employ "shorter delivery time frames, an approach consistent with Agile" when developing or acquiring IT. OMB data suggested Agile practices could help federal agencies and other organizations design and acquire software more effectively, but agencies needed to understand the risks involved in adopting these practices.
This blog post is the sixth in a series on Agile adoption in regulated settings, such as the Department of Defense, Internal Revenue Service, and Food and Drug Administration.
"Across the government, we've decreased the time it takes across our high-impact investments to deliver functionality by 20 days over the past year alone. That is a big indicator that agencies across the board are adopting agile or agile-like practices," Lisa Schlosser, acting federal chief information officer, said in a November 2014 interview with Federal News Radio. Schlosser based her remarks on data collected by the Office of Management and Budget (OMB) over the last year. In 2010, the OMB issued guidance calling on federal agencies to employ "shorter delivery time frames, an approach consistent with Agile" when developing or acquiring IT. As evidenced by the OMB data, Agile practices can help federal agencies and other organizations design and acquire software more effectively, but they need to understand the risks involved when contemplating the use of Agile. This ongoing series on Readiness & Fit Analysis (RFA) focuses on helping federal agencies and other organizations in regulated settings understand the risks involved when contemplating or embarking on a new approach to developing or acquiring software. Specifically, this blog post, the sixth in a series, explores issues related to system attributes organizations should consider when adopting Agile.
Tension and disconnects between software and systems engineering functions are not new. Grady Campbell wrote in 2004 that "systems engineering and software engineering need to overcome a conceptual incompatibility (physical versus informational views of a system)" and that systems engineering decisions can create or contribute to software risk if they "prematurely over-constrain software engineering choices" or "inadequately communicate information, including unknowns and uncertainties, needed for effective software engineering." This tension holds true for Department of Defense (DoD) programs as well, which historically decompose systems from the system level down to subsystem behavior and breakdown work for the program based on this decomposition. Hardware-focused views are typically deemed not appropriate for software, and some systems engineers (and most systems engineering standards) have not yet adopted an integrated view of the two disciplines. An integrated view is necessary, however, because in complex software-reliant systems, software components often interact with multiple hardware components at different levels of the system architecture. In this blog post, I describe recently published research conducted by me and other members of the SEI's Client Technical Solutions Division highlighting interactions on DoD programs between Agile software-development teams and their systems engineering counterparts in the development of software-reliant systems.
More and more, suppliers of software-reliant Department of Defense (DoD) systems are moving away from traditional waterfall development practices in favor of agile methods. As described in previous posts on this blog, agile methods are effective for shortening delivery cycles and managing costs. If the benefits of agile are to be realized effectively for the DoD, however, personnel responsible for overseeing software acquisitions must be fluent in metrics used to monitor these programs. This blog post highlights the results of an effort by researchers at the Carnegie Mellon University Software Engineering Institute to create a reference for personnel who oversee software development acquisition for major systems built by developers applying agile methods. This post also presents seven categories for tracking agile metrics.
As part of an ongoing effort to keep you informed about our latest work, I would like to let you know about some recently published SEI technical reports and notes. These reports highlight the latest work of SEI technologists in assuring software reliability, future architectures, Agile software teams, insider threat, and HTML5. This post includes a listing of each report, author(s), and links where the published reports can be accessed on the SEI website.
Federal agencies depend on IT to support their missions and spent at least $76 billion on IT in fiscal year 2011, according to a report from the Government Accountability Office (GAO). The catalyst for the study was congressional concern over prior IT expenditures that produced disappointing results, including multimillion dollar cost overruns and schedule delays measured in years, with questionable mission-related achievements. The Office of Management and Budget (OMB) in 2010 issued guidance that advocates federal agencies employ "shorter delivery time frames, an approach consistent with Agile." This ongoing series on the Readiness & Fit Analysis (RFA) approach focuses on helping federal agencies and other organizations understand the risks involved when contemplating or embarking on the adoption of new practices, such as Agile methods. This blog posting, the fifth in this series, explores the Practices category, which helps organizations understand which Agile practices are already in use to formulate a more effective adoption strategy.
As part of our mission to advance the practice of software engineering and cybersecurity through research and technology transition, our work focuses on ensuring that software-reliant systems are developed and operated with predictable and improved quality, schedule, and cost. To achieve this mission, the SEI conducts research and development activities involving the Department of Defense (DoD), federal agencies, industry, and academia. As we look back on 2013, this blog posting highlights our many R&D accomplishments during the past year.
Government agencies, including the departments of Defense, Veteran Affairs, and Treasury, are being asked by their government program office to adopt Agile methods. These are organizations that have traditionally utilized a "waterfall" life cycle model (as epitomized by the engineering "V" charts). Programming teams in these organizations are accustomed to being managed via a series of document-centric technical reviews that focus on the evolution of the artifacts that describe the requirements and design of the system rather than its evolving implementation, as is more common with Agile methods. Due to these changes, many struggle to adopt Agile practices. For example, acquisition staff often wonder how to fit Agile measurement practices into their progress tracking systems.
Agile projects with incremental development lifecycles are showing greater promise in enabling organizations to rapidly field software compared to waterfall projects. There is a lack of clarity, however, regarding the factors that constitute and contribute to success of Agile projects. A team of researchers from Carnegie Mellon University's Software Engineering Institute, including Ipek Ozkaya, Robert Nord, and myself, interviewed project teams with incremental development lifecycles from five government and commercial organizations. This blog posting summarizes the findings from this study to understand key success and failure factors for rapid fielding on their projects.
In our work with the Department of Defense (DoD) and other government agencies such as the U.S. Department of Veteran Affairs and the U.S. Department of the Treasury, we often encounter organizations that have been asked by their government program office to adopt agile methods. These are organizations that have traditionally utilized a "waterfall" life cycle model (as epitomized by the engineering "V" charts) and are accustomed to being managed via a series of document-centric technical reviews that focus on the evolution of the artifacts that describe the requirements and design of the system rather than its evolving implementation, as is more common with agile methods.
In launching the SEI blog two years ago, one of our top priorities was to advance the scope and impact of SEI research and development projects, while increasing the visibility of the work by SEI technologists who staff these projects. After 114 posts, and 72,608 visits from readers of our blog, this post reflects on some highlights from the last two years and gives our readers a preview of posts to come.
The adoption of new practices, such as agile or any new practice for that matter, is a task that is best undertaken with both eyes open. There are often disconnects between the adopting organization's current practice and culture and the new practices being adopted. This posting is the second installment in a series on Readiness & Fit Analysis (RFA), which is a model and method for understanding risks when contemplating or embarking on the adoption of new practices, in this case agile methods.
The Department of Defense (DoD) has become deeply and fundamentally reliant on software. As a federally funded research and development center (FFRDC), the SEI is chartered to work with the DoD to meet the challenges of designing, producing, assuring, and evolving software-reliant systems in an affordable and dependable manner. This blog post--the first in a multi-part series--outlines key elements of the forthcoming SEI Strategic Research Plan that addresses these challenges through research and acquisition support and collaboration with DoD, other federal agencies, industry, and academia.
All software engineering and management practices are based on cultural and social assumptions. When adopting new practices, leaders often find mismatches between those assumptions and the realities within their organizations. The SEI has an analysis method called Readiness and Fit Analysis (RFA) that allows the profiling of a set of practices to understand their cultural assumptions and then to use the profile to support an organization in understanding its fit with the practices' cultural assumptions. RFA has been used for multiple technologies and sets of practices, most notably for adoption of CMMI practices.
This post is the third and final installment in a three-part series that explains how Nedbank, one of the largest banks in South Africa, is rolling out the SEI's Team Software Process (TSP) throughout its IT organization. In the first post of this series, I examined how Nedbank addressed issues of quality and productivity among its software engineering teams using TSP at the individual and team level. In the second post, I discussed how the SEI worked with Nedbank to address challenges with expanding and scaling the use of TSP at an organizational level. In this post, I first explore challenges common to many organizations seeking to improve performance and become more agile and conclude by demonstrating how SEI researchers addressed these challenges in the TSP rollout at Nedbank.
While agile methods have become popular in commercial software development organizations, the engineering disciplines needed to apply agility to mission-critical, software-reliant systems are not as well defined or practiced. To help bridge this gap, the SEI recently hosted the Agile Research Forum. The event brought together researchers and practitioners from around the world to discuss when and how to best apply agile methods in mission-critical environments found in government and many industries. This blog posting, the fifth and final installment in a multi-part series highlighting research presented during the forum, summarizes a presentation I gave on the importance of applying agile methods to common operating platform environments (COPEs)that have become increasingly important for the Department of Defense (DoD).
While agile methods have become popular in commercial software development organizations, the engineering disciplines needed to apply agility to mission-critical, software-reliant systems are not as well defined or practiced. To help bridge this gap, the SEI recently hosted the Agile Research Forum. The event brought together researchers and practitioners from around the world to discuss when and how to best apply agile methods in mission-critical environments found in government and many industries.
While agile methods have become popular in commercial software development organizations, the engineering disciplines needed to apply agility to mission-critical, software-reliant systems are not as well defined or practiced. To help bridge this gap, the SEI recently hosted the Agile Research Forum. The event brought together researchers and practitioners from around the world to discuss when and how to best apply agile methods in mission-critical environments found in government and many industries. This blog posting, the third installment in a multi-part series highlighting research presented during the forum, summarizes a presentation made during the forum by Ipek Ozkaya, a senior researcher in the SEI's Research, Technology & System Solutions program, who discussed the use of agile architecture practices to manage strategic, intentional technical debt.
While agile methods have become popular in commercial software development organizations, the engineering disciplines needed to apply agility to mission-critical, software-reliant systems are not as well defined or practiced. To help bridge this gap, the SEI recently hosted the Agile Research Forum, which brought together researchers and practitioners from around the world to discuss when and how to best apply agile methods in mission-critical environments found in government and many industries. This blog posting, the second installment in a multi-part series, summarizes a presentation made during the forum by Mary Ann Lapham, a senior researcher in the SEI's Acquisition Support Program, who highlighted the importance of collaboration with end users, as well as among cross-functional teams, to facilitate the adoption of agile approaches into DoD acquisition programs.
While agile methods have become popular in commercial software development organizations, the engineering disciplines needed to apply agility to mission-critical software-reliant systems are not as well defined or practiced. To help bridge this gap, the SEI recently hosted the Agile Research Forum, which brought together researchers and practitioners from around the world to discuss when and how to best apply agile methods in the mission-critical environments found in government and many industries. This blog posting, the first in a multi-part series, highlights key ideas and issues associated with applying agile methods to address the challenges of complexity, exacting regulations, and schedule pressures that were presented during the forum.
New acquisition guidelines from the Department of Defense (DoD) aimed at reducing system lifecycle time and effort are encouraging the adoption of Agile methods. There is a general lack, however, of practical guidance on how to employ Agile methods effectively for DoD acquisition programs. This blog posting describes our research on providing software and systems architects with a decision making framework for reducing integration risk with Agile methods, thereby reducing the time and resources needed for related work.
The appeal of Agile or lightweight development methods has grown steadily in the software development community. Having spent a number of years investigating strategic planning approaches, I've recently been thinking about whether Agile principles can be--and should be--applied to strategic planning. This blog post examines the applicability of Agile principles to strategic planning.
We use the SEI Blog to inform you about the latest work at the SEI, so this week I'm summarizing some video presentations recently posted to the SEI website from the SEI Technologies Forum. This virtual event held in late 2011 brought together participants from more than 50 countries to engage with SEI researchers on a sample of our latest work, including cloud computing, insider threat, Agile development, software architecture, security, measurement, process improvement, and acquisition dynamics. This post includes a description of all the video presentations from the first event, along with links where you can view the full presentations on the SEI website.
Over the past several years, the SEI has explored the use of Agile methods in DoD environments, focusing on both if and when they are suitable and how to use them most effectively when they are suitable. Our research has approached the topic of Agile methods both from an acquisition and a technical perspective. Stephany Bellomo described some of our experiences in previous blog posts What is Agile? and Building a Foundation for Agile. This post summarizes a project the SEI has undertaken to review and study Agile approaches, with the goal of developing guidance for their effective application in DoD environments.
Managing technical debt, which refers to the rework and degraded quality resulting from overly hasty delivery of software capabilities to users, is an increasingly critical aspect of producing cost-effective, timely, and high-quality software products. A delicate balance is needed between the desire to release new software capabilities rapidly to satisfy users and the desire to practice sound software engineering that reduces rework.
A key mission of the SEI is to advance the practice of software engineering and cyber security through research and technology transition to ensure the development and operation of software-reliant Department of Defense (DoD) systems with predictable and improved quality, schedule, and cost. To achieve this mission, the SEI conducts research and development (R&D) activities involving the DoD, federal agencies, industry, and academia. One of my initial blog postings summarized the new and upcoming R&D activitieswe had planned for 2011. Now that the year is nearly over, this blog posting presents some of the many R&D accomplishments we completed in 2011.
Occasionally this blog will highlight different posts from the SEI blogosphere. Today's post is from the SATURN Network blog by Nanette Brown, a senior member of the technical staff in the SEI's Research, Technology, and System Solutions program. This post, the third in a series on lean principles and architecture, continues the discussion on the eight types of waste identified in Lean manufacturing and how these types of waste manifst themselves in software development. The focus of this post is on mapping the waste of motion and the waste of transportation from manufacturing to the waste of information transformation in software development.
As industry and government customers demand increasingly rapid innovation and the ability to adapt products and systems to emerging needs, the time frames for releasing new software capabilities continue to shorten. Likewise, Agile software development processes, with their emphasis on releasing new software capabilities rapidly, are increasing in popularity beyond their initial small team and project context. Practices intended to speed up the delivery of value to users, however, often result in high rework costs that ultimately offset the benefits of faster delivery, especially when good engineering practices are forgotten along the way. This rework and degrading quality often is referred to as technical debt. This post describes our research on improving the overall value delivered to users by strategically managing technical debt, which involves decisions made to defer necessary work during the planning or execution of a software project.
This is a second in a series of posts focusing on Agile software development. In the first post, "What is Agile?" we provided a short overview of the key elements of the Agile approach, and we introduced the Agile Manifesto. One of the guiding principles from the manifesto emphasizes valuing people over developing processes. While the manifesto clearly alludes to the fact that too much focus on process (and not results) can be a bad thing, we introduce the notion here that the other end of the spectrum can also be bad. This blog explores the level of skill that is needed to develop software using Agile (do you need less skill or more?), as well as the importance of maintaining strong competency in a core set of software engineering processes.
If you ask the question, "What is Agile?" you are likely to get lots of different answers. That's because there is no universally accepted formal definition for Agile. To make matters worse, there are ongoing debates over what Agile software development SHOULD mean. That being the case, when answering the question, "What is Agile?" the safest bet is to stick to what people can agree on, and people generally agree on three key elements of Agile. Taken together, these describe the Agile software development method, as well as the software development approach. In this post--the first in a series on Agile--I will explain the foundations of Agile and its use by developers.
There's been a widespread movement in recent years from traditional waterfall development to Agile approaches in government software acquisition programs. This transition has created the need for personnel who oversee government software acquisitions to become fluent in metrics used to...