Taking DevSecOps to the Next Level with Value Stream Mapping

This post explores the relationship between DevSecOps and value stream mapping, both of which are rooted in the Lean approach to systems and workflow. It also provides guidance on preparing to conduct value stream mapping within a software-intensive product development environment.

If the focus of post-waterfall software engineering could be summed up in one word, it would be flow, which focuses on reducing the time for items of customer value (e.g., features) to move from concept to deployment. Lean software development, DevSecOps, and value stream management all consciously orient their principles and practices around flow optimization. Although Agile software methods don’t often mention flow explicitly, flow optimization is implicit in Agile’s focus on the incremental delivery of value and the use of empowered, cross-functional teams to minimize impediments and delays.

Flow is an intuitively accessible concept. Rivers flow unless impeded by dams or rock formations. Our minds in a state of flow are unimpeded, focused, and energized. Software development is not concerned with the flow of water or internal consciousness but rather with the flow of value to customers and end users. By focusing on flow, we aim to achieve value as soon as possible and to eliminate any impedance or friction. Iterative and incremental development, continuous integration and delivery, minimum viable product, and minimum viable capability release all have the rapid flow of value as their raison d’etre.

A focus on flow underlies and unifies the topics discussed in this post. Value streams and DevSecOps are rooted in the premise that organizational boundaries should be subsumed in the pursuit of flow. Value stream mapping provides a framework for identifying existing barriers to flow and designing a future state in which value flows more freely.

Lean and Value Stream Origins

Lean as a philosophy and set of practices was popularized in a 1990 book about Toyota and the Toyota Production System (TPS), The Machine That Changed the World, by Womack, Roos, and Jones. TPS is a philosophy and set of practices adopted by Toyota in the years following World War II and influenced by the work of W. Edwards Deming. In Lean Thinking, published in 1996, Womack and Jones refined their earlier work, identifying five Lean principles, one of which was value stream. In 1998, Rother and Shook wrote Learning to See, which provided detailed guidance on how to perform value stream mapping, which they based upon a TPS practice called Material and Information Flow Mapping.

A passion for optimizing flow is a hallmark of Lean, as illustrated by this passage in Lean Thinking:

When we start thinking about ways to line up all of the essential steps needed to get a job done into a steady, continuous flow, with no wasted motions, no interruptions, no batches, no queues, it changes everything: how we work together, the kinds of tools we devise to help with our work, the organizations we create to facilitate the flow, the kinds of careers we pursue, the nature of business firms (including nonprofit service providers) and their linkages to each other and society.

Value Streams and Value Stream Mapping

In Project to Product, Mik Kersten defines a value stream as

the end to end set of activities performed to deliver value to a customer through a product or service… Value streams are composed of all the activities, stakeholders, processes & tools required to deliver business value to the customer

Value stream mapping is a Lean technique for visualizing, characterizing, and continuously improving the flow of value across this set of end-to-end activities by eliminating barriers, whether procedural, cultural, technological, or organizational. In the Lean literature, these barriers are referred to as “waste.” Common examples of waste in software development organizations include unnecessary waiting or task switching, adding extra features that yield software bloat, and duplicating code in violation of the “don’t repeat yourself” (DRY) principle.

While the practice is referred to as value stream mapping, this is something of a misnomer. While value stream maps are indeed created and revised, the focus of the practice is to enable improved, ongoing value stream management through the application of Lean process improvement techniques and metrics. The purpose of creating a current-state value stream map is to establish the process understanding and data required to successfully evolve to an improved future state. For example, mapping a software development value stream might reveal unnecessary handoffs between functional organizations, overly complex approval processes, long testing delays due to manual processes, etc.

The DevOps–Lean Connection

DevOps is a software development concept that literally and figuratively blends development and operations staff and tools. There’s a deep connection between DevOps and Lean methods. In fact, The DevOps Handbook describes DevOps as “the result of applying Lean principles to the technology value stream.” The technology value stream is “the process required to convert a business hypothesis into a technology-enabled service that delivers value to the customer.” That process includes product management, development, quality assurance, IT operations, and information security. The DevOps Handbook focuses on the application of Lean principles to a subset of the technology value stream, which “begins when any engineer in our value stream…checks a change into version control and ends when that change is successfully running in production.”

The DevOps–Value Stream Mapping Connection

DevOps and value streams are both rooted in Lean principles. Both are being increasingly employed to accelerate software development agility. However, there are different perspectives among thought leaders regarding how they relate to one another. This section characterizes a few of these perspectives.

The DevOps Handbook lists “The Principles of Flow” as the first of the “Three Ways” of DevOps and discusses using value stream mapping as a tool to optimize flow across the DevOps subset of the technology value stream. The perspective in the handbook is that value stream mapping helps DevOps teams be more successful at optimizing their flow through the application of Lean thinking and tools such as flow visualization, flow-based metrics, and identification of waste and non-value-added activities.
Since the publication of The DevOps Handbook, DevOps has been expanded to DevSecOps to include security and Dev*Ops to cover other elements to be incorporated into the automated pipeline, such as safety. In addition, there has been a growing emphasis on the need to establish robust linkages between DevSecOps and the totality of the technology value stream. Gartner’s Market Guide for DevOps Value Stream Management Platforms describes a perspective whereby value stream mapping helps DevSecOps teams be more successful at supporting the enterprise by expanding their focus “beyond siloed operational metrics to instead deliver on customer-centric, team-level performance indicators.”
As interest in value streams continues to grow, some argue that advances in software development and delivery require that DevSecOps be subsumed within a focus on the larger end-to-end value stream. This perspective, as expressed in this Software Development Times article, contends that Agile and DevOps haven’t fully delivered on their promises because they are confined to the right-most segment of the overall flow of value through the enterprise. It sees value stream mapping as the next frontier, applying the foundational principles Agile and DevSecOps to focus on accelerating value across the entire enterprise.

Regardless of perspective, this post seeks to provide insight into applying value stream mapping for DevSecOps practitioners and all members of the technology value stream.

Adopting and Adapting Value Stream Mapping—Leveraging Experience, Respecting Differences

For decades, Lean manufacturing practitioners have honed and improved their value stream mapping practices. The manufacturing approach to value stream mapping currently provides a mature framework and a set of lessons learned. However, the important distinctions between manufacturing and software-intensive product development impact how value stream mapping is conducted both conceptually (e.g., What is waste?) and procedurally (e.g., Which mapping icons should be utilized?).

This blog post and future posts seeks to leverage the maturity of the manufacturing approach to value stream mapping while incorporating adaptations required to apply this practice to software-intensive product development. It will draw on Value Stream Mapping by Martin and Osterling for its discussion of value stream mapping in the manufacturing domain and Project to Product for its application of value stream mapping concepts to software development.

The following provides an overview of key distinctions between the domains of manufacturing and software-intensive product development.

Discovery as well as Execution

In manufacturing, value stream mapping is applied to the process space that transforms raw material and subcomponents from suppliers into finished goods for customers. This process space focuses almost exclusively on production-oriented activities as well as on executing and delivering to a finalized specification.

In software development, the mapping exercise is typically applied to a development value stream tasked with the delivery of a product or system. The product or system is in turn being developed to support or enable an associated operational value stream. Rather than being exclusively focused on execution, the development process space (particularly in Agile development) interleaves discovery-oriented activities (requirements refinement, architecture, technical spikes, exploratory testing) with production-oriented activities (build, configuration management, regression test, coding to well-established design patterns or mature requirements).

Variance in Relationship to Value

Both Lean manufacturing and Lean product development seek to reduce time to value (i.e., the time between a user’s request and their experience of the requested value in the form of a delivered product). However, because the nature of their process spaces differs, the means of reducing time to value will differ for manufacturing and development value streams.

Lean Manufacturing focuses on eliminating process variation and waste to optimize flow and time to value. In Lean manufacturing the question What constitutes value? has been asked and answered prior to entering the value stream process space.

In software-intensive product development, particularly in an Agile development context, the definition of value is ongoing and evolving. A Lean approach to software-intensive product development must address the goal of accelerating knowledge acquisition. By its nature, the pursuit of knowledge encompasses a degree of unpredictability, so it introduces variance into the workflow. Moreover, the intangible nature of product development work items makes strict standardization difficult and means that a certain degree of variation is inherent in the domain. A Lean approach to software-intensive product development must accommodate variance in ways that manufacturing does not.

Information as Material

In the manufacturing domain, a value stream map tracks the flow of raw materials through the process of being transformed into a finished good that provides value to the customer, which is known as the material flow. In addition to the material flow, manufacturing value stream maps represent an information flow. The information flow tracks the information and systems that interact with, support, and schedule material flow processing.

In manufacturing there is a clear distinction between the tools that perform physical transformations along the material flow and the digital systems and data represented in the information flow. In software-intensive product development, the distinction is less obvious as information is used both as a raw material and as a support mechanism.

Preparing for Value Stream Mapping

We now turn our focus on how to prepare to conduct a value stream mapping effort, which involves the following two activities.

Selecting the value stream mapping team—Value stream mapping is intended to be a team activity for two main reasons. First, it is unlikely that any one individual will have the knowledge and insight to construct an accurate representation of the value stream. Second, the purpose of the value stream exercise is to identify improvement opportunities and effect change. This can only be brought about by the engagement of value stream participants and stakeholders.

The value stream mapping team should be small enough for close collaboration and should collectively provide sufficient knowledge and expertise to map the current state of the value stream, collect metrics, identify improvement areas, and design the desired future state.

The presence of a value stream manager or champion, either as a member or as a sponsor of the mapping team is also critical. As stated above, the purpose of value stream mapping is to effect change. The value stream manager or champion should have the ability to authorize, enable, socialize, and provide resources in support of future-state value stream design as well as monitor and track its implementation. The needed resources will differ depending on the value stream being mapped, but generally speaking they may include funding, development staff, training and coaching, tools and technology, re-negotiation of inter-department work flow, and so on.
Chartering the value stream mapping effort—Chartering provides the value stream mapping team with clarity concerning the processes they will be mapping and the value these processes are expected to deliver to stakeholders. It also provides the team the opportunity to agree on vocabulary, discuss measurement approaches, and so on. Some organizations might adopt a formal charter document. Other organizations may use a wiki or even a board with sticky notes to document common understanding. The most important outcome of chartering is to ensure a clear and common foundational point for all team members and to do so in a way that allows this foundation point to evolve as knowledge is gained.

For the sake of brevity, this initial foundation of understanding is referred to below as the
charter, which should encompass the following:

A description of the value stream, its stakeholders, and the value they expect to receive—A description of the full value stream to be mapped (from value request through value fulfillment) should be included in the charter to ensure a clear understanding and focus on the creation of value for the customer. The description should include a discussion of the value stream stakeholders, their demands upon the value stream, their expectations for value delivery, and the extent to which these expectations are being met. These details enable the value stream mapping team to orient their analysis and improvements around the stakeholders’ need for value. In many cases, achieving the goals of the current value stream mapping effort might not require that the full value stream be mapped. Under these circumstances, the first and last steps of the value stream segment to be mapped should be identified.

The rationale for conducting the mapping exercise—The charter should clearly state the reasons for undertaking the value stream mapping exercise. What challenges and issues are stakeholders experiencing? What improvements would they like to see? For software-intensive product development, it is also valuable to understand the mix of discovery/design and execution-oriented activities currently being undertaken in the value steam or value stream segment. If discovery/design is the predominant concern, optimizations that accelerate knowledge acquisition will be the focus. If execution is the predominant concern, value stream mapping efforts will typically focus on reducing waste and improving predictability.

In addition to understanding what improvements stakeholders would like to see, it is also important to gain a rough understanding of when they would like to see them. Establishing desired improvement timeframes can help keep the team focused on stakeholder goals and protect against having the value stream map become overly detailed or a goal in and of itself. As a Lean practice, value stream mapping presumes that value stream improvements will be attained iteratively via a plan-do-check-act (PDCA) cycle. Therefore, improvement timeframes should encompass both short-term and long-term horizons and allow for goals to be revisited and revised.

Definition of critical terms and concepts—In Lean manufacturing, “raw materials are pulled (by customer orders)…through the production process that converts them to finished goods that deliver value to the client. Value stream mapping in the manufacturing domain seeks to optimize this flow of raw materials.

To perform value stream mapping for software-intensive products, we need comparable conceptual clarity regarding what flows through the value stream and how these items relate to customer value. In Project to Product, Mik Kersten introduces the concept of a flow item, which he defines as “a unit of business value pulled by a stakeholder through a product’s value stream.”
Based upon an analysis of 308 enterprise IT tool networks, Kersten identified five flow items that characterize software-intensive product development:
- features that deliver new business value to customers 
- defect resolutions that deliver quality to customers
- identified risks that, when resolved, deliver security
- governance and compliance to security and risk officers
- technical debts that remove impediments to future delivery for architects
Kersten’s book provides further details on these flow items and presents a comprehensive flow framework. Whether or not an organization chooses to adopt this particular flow item taxonomy, identifying and defining the flow items to track is an imperative step towards achieving value stream mapping goals.

Value stream mapping seeks to minimize time to value (i.e., the time elapsed between the expression of a customer need and when that need is fulfilled). Defining flow items provides clarity around the value that a value stream is intended to deliver. It is equally important to clearly define terms related to time. Even in the mature world of manufacturing, seemingly authoritative sources may differ in their applications of the terms cycle time and lead time. It is critical that members of the value stream mapping team agree to a common vocabulary with respect to the measurement of time. Time-related concepts that may require labeling (depending upon your application) include the following:
- elapsed time to complete all value stream processes from initiation to completion of the finished good
- elapsed time to traverse a value stream segment (i.e., to complete the processes within that segment and be made available to the next process step)
- elapsed time from customer-initiated request to fulfillment
Note that value stream mapping makes a critical distinction between the time a work item is being actively processed (touch time, active time, process time) and elapsed time, which includes wait time. Reduction of wait time and improvement of flow efficiency (ratio of active time to total elapsed time) is a key goal of value stream mapping.
Measurement approach—Details of the measurement approach will likely evolve as value stream mapping activities proceed and as improvements are implemented. However, documenting the team’s initial thoughts on measurement collection may be a useful addition to the charter. Potential information that may be captured includes the following:
- the use of qualitative versus quantitative data
- anticipated availability of/effort to obtain quantitative data
- resources required to support measurement approach
Team constraints and degrees of freedom—The value stream mapping charter should document the avenues for change that are open to the value stream mapping team as well as any constraints they must respect. For example, there may be financial or organizational barriers to investing in tooling or in relocating remote staff. Identifying and socializing these boundaries is an important preparatory activity that ensures that the value stream mapping team is not working under assumptions that artificially shrink (or enlarge) the future solution space.

After preparations are complete and the foundations for the value stream mapping effort have been laid, the next step is to create the current-state value stream map, which will be the focus of a future post on this topic.

Software Engineering Institute

SEI Blog