A longstanding challenge in large software-reliant systems has been to provide system stakeholders with visibility into the status of systems as they are being developed. Such information is not always easy for senior executives and others in the engineering path to acquire when needed. In this blog post, we present a case study of an SEI project in which digital engineering is being used successfully to provide visibility of products under development from inception in a requirement to delivery on a platform.

One of the standard conventions for communicating about the state of an acquisition program is the program management review (PMR). Due to the accumulation of detail presented in a typical PMR, it can be hard to identify tasks that are most urgently in need of intervention. The promise of modern technology, however, is that a computer can augment human capacity to identify counterintuitive aspects of a program, effectively increasing its accuracy and quality. Digital engineering is a technology that can

• increase the visibility of what is most urgent and important;
• identify how changes that are introduced affect a whole system, as well as parts of it; and
• enable stakeholders of a system to retrieve timely information about the status of a product moving through the development lifecycle at any point in time.

In the SEI blog post Some Challenges in Making the Transition to Digital Engineering, Bill Nichols provided the following summary of digital engineering:

Digital engineering uses digital tools and representations in the process of developing, sustaining, and maintaining systems, including requirements, design, analysis, implementation, and test…digital engineering is well suited to the DoD’s need to sustain and maintain long-living systems whose missions evolve over time. The digital modeling approach is intended to establish an authoritative source of truth (ASOT) for the system in which discipline-specific views of the system are created using the same model elements. This model-based approach carries forward into the design and implementation.

A digital modeling environment effectively applies model-based systems engineering (MBSE) to the design and creates a common standards-based approach to documenting a system that enforces the use of standards by all stakeholders. A common modeling environment with commonly accepted and well-defined properties and stereotypes is intended to improve the ability to analyze the system and reduce the likelihood of finding late defects…Ideally, with MBSE this information can be stated once and then automatically propagated to various views of the data for all stakeholders. The result of this approach is an overall reduction of development risks, the ability to find and correct defects earlier in development when changes are relatively inexpensive, and elimination of document-driven development.

Through MBSE, digital engineering can provide visibility into design decision-making to those who are closest to the work. Developers can ask that more detailed and accurate information about the system and its behavior be provided to them through MBSE and the model. They can use MBSE and the model to provide more formal feedback to architects and systems engineers.

By providing visibility into the status of systems under development, MBSE has the potential to overcome the problem of uninformed decisions being made at higher levels of the hierarchy that then negatively affect the developers and the development. Figure 1 provides an example indicating relationships among model elements that, when populated, will show important bindings among activities in the development. It is these relationships that bring design and programmatic decisions to light with many system stakeholders, enabling discussion and reconsiderations, and helping to prevent design blind alleys.

Figure 1 shows high-level architectural elements and relationships among those elements as part of the fundamental definition of a roadmap model. The elements and the relationships were developed in a process of continuous communication among an engineering leadership core having relevant experience and authority in the dimensions represented in this overall model construct. In subsequent steps, relations to specific instances contained within these more general elements are correlated, with corresponding dialogue and factoring to ensure coherence in the model as understanding deepens.

Our SEI team is currently working on replacement of a U.S government enterprise-planning system that had been in operation for almost 20 years. The new system utilizes a microservices architecture and a cloud-based platform. A goal for the new system has been to apply an open, transparent architectural approach that supports the needs of multiple enterprise-planning services by making the system fully interoperable.

Our project built upon the concept of using MBSE to provide decision support to a variety of more than 20 stakeholders. We expect that senior leaders, engineers at all levels, systems specialists, testers, and others who have specific questions about the system as it is incrementally built and delivered will be able to make queries of the model more easily to determine the status of components in development and to answer questions such as

• How soon can I expect component X to be done?
• What will happen if we introduce this extra capability at this time? (i.e., what would be the ripple effect? Digital engineering makes visible the consequences of such changes in a way that they are not visible in the development practices that are common today.)
• I introduced a new work package. What is the status of that work package? The digital engineering systems should be able to provide an answer such as, it’s at this point in the development process, and we’re looking for a delivery date sometime around [some date].

Answers to these and similar questions will be supported by data in the model or family of models that document different aspects of the system, and by visibility into the changes that have occurred in the time between initial requirements and the current state. We expect that these models will be updated and informed by the as-built system and that the developers will use the models to make decisions about the order and sequence of their own work.

SEI staff members had previously used an MBSE environment called No Magic Cameo Enterprise Architecture (Cameo EA) successfully on another project. We therefore brought this experience and expertise to the project. Program planning is currently guided by a roadmap, such as the one shown in Figure 2 above, that depicts planned products along a timeline with intended delivery dates for each. Such a timeline cannot account for the complexity of how the planned products will be produced within the development environment. A digital development environment, however, instrumented with MBSE, can potentially help in the management of complexity by providing visibility into product status and dependencies with other products during development.

A great product requires the process infrastructure necessary to create it. Among the capabilities to establish in a digital engineering enterprise, MBSE is the organizational activity responsible for modeling of requirements, architecture, and design of a product. The goal is to support the enabling factory structure that best implements efficient delivery of a sufficiently high-quality product.

The vision for applying digital engineering in this project is that it will provide visibility of products under development, from inception in a requirement to delivery. With the new system, answers to questions that typically arise in a PMR will be available all the time. In particular, time-consuming inquiries will be reduced to a routine set of answers readily available in 24 hours or less, which means that

• The current practices must be understood.
• The elements of the current practices have been mapped and modeled.
• Opportunities for efficiency among current practices (considering opportunities for technology insertion) must be described, understood, and tested.
• All legacy data must have been ported from retired software to our new models in the Cameo EA and all elements of the data model reconciled so that we have a standard for data capture and input for all prospective stakeholders.

The goal of our work is to create a development pipeline that accounts for the constant evolution of incoming requirements. Requirements can come from a multitude of formal or informal sources. Digital engineering provides a way to account for any transformations that the requirement has gone through as it has been decomposed and has become part of the pipeline, and at any given time (see Figure 2).

Over the course of this engagement, we have supported the fundamental idea of holism. In particular, our efforts should capture the whole development–delivery pipeline from the inception of ideas to the delivery of components to platform. In initial discussions with our sponsor, we discussed how the need for an authoritative source of truth should be reflected in requirements. This dialogue led to talks and presentations, and the work has progressed steadily since then with a continuous focus toward bringing real answers about the development to decision-makers on the program.

The process of building the model involves developing schemas: diagrams (see Figure 1) describing the model elements that interact among areas of focus. These areas of focus include the capabilities to be built as well as the relationships among the business model architecture elements, operations-to-business behavior, and systems services and resources to model the solution architecture. Our team is currently working on the foundational activity to bring stakeholders to a common schema representation suitable to begin ingesting current program data. The structure of this schema will determine the utility of the modeling tool. Since the schema will be a stakeholder-derived product, it will establish the model as the authoritative source of truth and make data labels common across all platforms and operational value streams.

All these attributes have been in play on this project, with the best part of our engagement being that we can demonstrate and support existing practices, but with methods and system characteristics that nearly every circumspect engineer can appreciate as sound. Among these are: accountability, measurability, increased status visibility, information as a better basis for decision-making, and the ability to control critical aspects of the systems and software engineering environment.

Highlights of the SEI team’s work thus far include

• creating an algorithm to move all the data into our current modeling environment in Cameo EA
• designing and regularly shaping the receiving Cameo EA modeling schema to accommodate the many data irregularities that are typical in any database-transfer activity
• ensuring that we tie these system capabilities to a strong vision of future capability that anticipates change and serves the need of flexibility over time.

This engagement has taught us all a great deal about the nature of change and technology: Nearly all technological advances require effort to build support for integration with legacy processes. Everyone involved must exercise patience since it takes time for people in roles affected by transitions in which aspects of work will be altered to consider how the new situation will help. A solution must have demonstrated efficacy and safety before it can become part of deployment to the public.

In the domain that we are supporting—enterprise planning—the number of participating platforms and their stakeholder groups is large. The challenges of developing a solution that supports the needs of these communities are steep, and yet the diversity of interests forces us to examine and understand the needs of these groups and to develop the solution in answer to those needs. Any other approach risks imposing solutions that do not adequately address these needs. Moreover, by cross comparison, we discover which elements among these expressed needs are common, unique, or shared. The nature of our work hovers closely within these considerations due to the need for clear separation of concerns—a core concept from systems engineering, applied in the development of this systems engineering tool.

The world is rightly concerned with the power of artificial intelligence in many aspects of military capability. This focus could well be prioritized as the next consideration on our project. Computers have the distinct advantage, even over teams of people, of being able to identify and therefore reveal gaps among many interacting elements. Receiving advice from artificial intelligence therefore becomes possible after many of the critical elements in the chain of production are automated.

The development critical chain is often complex, and answers to questions of priority among components and related capabilities must be considered in ways that bring opportunities and risks in processes to the awareness of decision-makers. This process is similar to how a relatively simple matrix of a chessboard hides the many possibilities that remain unseen by many chess players. Artificial intelligence stands to even this score for decision-makers and to avoid critical missteps hiding among the many levels of complexity that far exceed those of any game, and whose stakes are well known to all.