The first blog post in this series provided an overview of the E2E Mission Thread Data Flow Analysis (EMDA) method, an approach that analyzes the flow of data as they traverse end-to-end mission threads through the architecture components of a system of systems. That post addressed relevant challenges that EMDA helps system and software architects face and outlined the work products produced by the method. This second blog post discusses the process used to create and verify the method's work products, the benefits of the method, the challenges must be addressed while implementing the method, and lessons learned during the use of this method on a U.S. military program.

Process

The process for performing E2E data flow analysis consists of the following tasks, performed in a highly incremental and iterative manner:

1. Identify relevant sources of information. This process includes identifying documentation, such as system and software architecture documents, various architecture-related diagrams, whitepapers, and modeling tools/databases. Identifying information sources also includes identifying subject-matter experts (SMEs), such as system, software, and network architects; technical leaders; safety and security engineers; testers; and working groups, as well as stakeholders, such as operators and maintainers. Create the initial source documentation list and SME contact list. Maintain these lists, keeping them current for the duration of the analysis.
2. Collect access to source documentation. Obtain copies of diagrams, whitepapers, and architecture documents. Obtain read access to modeling tools, which may require authorization and a measure of politically sensitivity, especially for access to modeling tool databases belonging to external organizations. You may need to settle for snapshots from these databases instead of real-time access. The source documentation may be For Official Use Only (FOUO) or classified; if so, the resulting data flow diagrams (DFDs) will likely have require the same marking and classification.
3. Establish SME cooperation. If necessary, obtain authorization to communicate with the SMEs, especially those that are external to one's own organization. Authorization is especially critical and politically sensitive when communicating with prime contractors and their subcontractors. Take care not to accidentally task the contractors to do work, such as creating and supplying missing documentation.
4. Read source documentation. Read the source documentation, highlighting the relevant information and taking notes as appropriate.
5. Identify and select important E2E data flows. Identify the system-of-system (SoS)-level mission threads or system-level use cases to model using E2E DFDs. Make your identification and selection based on coverage of important (1) functional capabilities, (2) architectural components and the connections between them, (3) data flows that are mission-, safety-, or security-critical, and (4) areas of the architecture that are least understood and require clarification.
6. Create initial E2E DFDs. Start with a simple, straight-forward flow that (1) provides primary functionality and (2) occurs frequently according to the operational profile, and touches many of the relevant architectural components. This flow will increase productivity and produce a lot of information about nodes and arcs that can be used on later E2E DFDs. Create an initial strawman diagram to show the SMEs.
7. Interview SMEs. Interview the SMEs to obtain their inputs on the existing E2E DFDs. Bring hard copies of the diagram(s) so that the SMEs can mark them up. Take notes including (1) proposed modifications to make after the interview, (2) inconsistencies due to disagreements between SMEs, (3) relevant architectural decisions that have yet to be made, (4) additional SMEs to interview, and (5) new documentation to obtain and review.
8. Incrementally iterate E2E DFDs until complete. Use the notes from SME interviews and any new documentation to make additions and fixes to the evolving diagrams.
9. Analyze E2E DFDs. Use the E2E DFDs to analyze the SoS and system architectures for their support for interoperability, performance, robustness, safety, and security ramifications. For example, analyze the nodes and arcs for potential security vulnerabilities and places that lack adequate defense in depth.
10. Verify and validate the diagrams. Have the SMEs verify the completeness, consistency, and correctness of the E2E DFDs. Have the SMEs validate the ability of the SoS architecture (i.e., E2E data flows) to meet the relevant functional, interoperability, performance, robustness, safety, and security requirements.
11. Present results. Present the final diagrams and the results of their analysis to technical management and important stakeholders so that major risks are reported and changes to the architecture can be discussed and approved.
12. Update documentation. Incorporate the E2E DFDs into the system and software architecture documents as well as any other relevant documents. If a general drawing tool was used to produce the diagrams and a modeling tool exists, then input the diagrams into the tool.

Benefits of the EMDA Method

Using E2E DFDs to model mission thread or use case data flows through an SoS addresses the architecture engineering challenges listed in the previous blog. This approach also provides many advantages over the current practice of using multiple diagrams that have more limited scope and capture different viewpoints.

• Show big picture. E2E DFDs show system and software engineers and work groups how the architectural components for which they are responsible fit into the overall system and system of systems. They show how hardware and software interact (e.g., from sensors to controllers to actuators).
• Focus on actual behavior. By concentrating on specific mission threads, use cases, and associated E2E data flows, the E2E DFDs concentrate the SME's attention on the implementation of specific functionality. This focus contrasts with static architecture diagrams (e.g., network diagrams) that show connections between architectural components without showing what interactions they support or how the connections fit together to support specific functionality. Creating the E2E DFDs helps ensure than no important interactions accidentally slip through the cracks.
• Clarify component collaboration. E2E DFDs show all of the architectural components that must collaborate to provide a specific functional capability.
• Increase consensus and consistency. E2E DFDs promote consensus and consistency across people (e.g., SMEs and stakeholders), organizations (e.g., acquirers, prime contractors, and subcontractors), and groups (e.g., requirements, architecture, networks, testing, safety, and security).
• Increase communication. E2E DFDs encourage communication between individuals, groups, and organizations, thereby helping to break down stovepipes.
• Tie together other views. E2E DFDs integrate disparate information (e.g., data flows through hardware, software, and network components) from other diagrams having more limited scope (e.g., context diagrams, software layer diagrams, network diagrams, system block diagrams, and sequence diagrams). By providing context, the E2E DFDs support the verification of the consistency and correctness of those other diagrams.
• Identify missing components. E2E DFDs help system, software, and network architects identify missing architectural components.
• Support Model-Based System/Software Engineering (MBSE). The results of E2E data flow analysis can be incorporated into various system/software modeling tools, such as network modeling tools and tools supporting modeling in the System Modeling Language (SysML), Unified Modeling Language (UML), and Architecture Analysis and Design Language (AADL).
• Support performance analysis. E2E DFDs support performance analysis (i.e., response time, throughput, and latency) by enabling the (1) identification of bottlenecks, (2) E2E addition of path-segment-specific timing allocations, and (3) verification of the reasonableness of how timing budgets are allocated.
• Support reliability/robustness analysis. E2E DFDs encourage analysis of how faults and failures propagate from one architectural component to another and from one system to another. These DFDs support the analysis of architecture-based fault and failure tolerance and provides answers to the following questions:
  • What are the default and back-up communication paths?
  • If a system-external component hangs or crashes, how does that propagate back to the system under development?
  • What happens when the external component comes back up?
  • How is data integrity and consistency ensured?
• Support security analysis. E2E DFDs support security analysis by helping security engineers understand the SoS-level attack surface and identify missing architectural security controls (e.g., weaknesses that are potential vulnerabilities) and places needing additional defense in depth. They can show where data in transit are encrypted/decrypted or digitally signed. They can also indicate architectural components where data at rest are protected.
• Support anti-tamper (AT) analysis. E2E DFDs can support anti-tamper by documenting how critical program information (CPI) flows within and between systems in the SoS as well as any architectural AT controls used to protect the CPI-in-transit. Note that AT-specific E2E DFDs will be classified and must be protected accordingly.
• Support size estimation. Technical management can use E2E DFDs as inputs to the COSMIC Functional Size Measurement Analysis Method.
• Support testing. Testers can use E2E DFDs to develop software and system integration tests, system tests, SoS integration tests, and SoS tests.
• Support sustainment. E2E DFDs support maintenance and sustainment by clarifying the impact of architectural changes on the implementation of functional capabilities.

In summary, E2E DFDs provide architectural insights that are hard or impossible to obtain by looking at individual diagrams with more limited scope. By looking at individual trees, it is hard to see the overall size and structure of the forest, not to mention the many paths through the forest.

Method-Specific Challenges

Modeling the E2E flow of data through an SoS is a non-trivial undertaking, especially when multiple flows are needed to model a reasonable subset of its functional capabilities and cover all relevant nodes. The following is a list of challenges to address when performing E2E data flow analysis:

• Iterative development. Relevant diagrams, documents, and SMEs are subject to human error. SMEs will often have misunderstandings about E2E flows, especially with regard to areas outside of their expertise. Early versions of the E2E DFDs will typically contain errors that are discovered over time as these DFDs are validated by more and more SMEs. Therefore, it is impossible to create the E2E DFDs in a straightforward, sequential manner. Their development, verification, and validation by SMEs must happen in a highly iterative manner with numerous versions being produced and updated over time.
• Incremental development. Software architects must develop multiple E2E DFDs to cover all important mission threads, architectural components, connections between components, and data flows. Development, verification, and validation by SMEs must therefore happen incrementally.
• Large number of actions and interactions. Modeling a single complete E2E data flow through an SoS down to the configuration-item level may easily require correctly identifying several dozen actions and interactions.
• Modeling requires significant investment. Even when a practical subset of flows is selected, the amount of effort and time required for proper modeling (including verification, validation with SMEs, and sustainment) is significant. This challenge is exacerbated by the fact that relevant SMEs are typically already very busy with other tasks.
• Exhaustive modeling is not practical. Because far more SoS mission threads and use case paths exist than can be exhaustively modeled using E2E flows, an appropriate subset must be selected to maximize benefits while keeping within schedule and manpower constraints.
• Multiple sources of information. No single system or software architecture diagram or document is likely to show all relevant actions and interactions. Similarly, no single SME or working group (e.g., chief engineer, technical leader, system architect, software architect, network architect, communications engineer, system administrator, security engineer, and safety engineer) will be familiar with all of these actions and interactions. Different people and documents typically have only local views of the relevant architectures, and even when there is overlap, the information will be incomplete and sometimes inconsistent. Therefore, free access to all relevant documents and SMEs is required to ensure complete and correct modeling. Obtaining this access can be hard because these diagrams, documents, and SMEs will belong to different system-level projects.
• Most diagrams show static structure. Most existing architectural diagrams (e.g., context diagrams, network diagrams, software layer diagrams, class diagrams) show static structure rather than dynamic behavior and therefore show the potential for communication rather than the actual communication associated with a specific mission thread.
• Most diagrams have limited scope. Most existing architectural diagrams are restricted to a single system, whereas E2E DFDs cut across multiple systems within an SoS. Similarly, most diagrams are restricted to showing either hardware or software but not both. To create an E2E DFD, therefore, information from disparate and potentially inconsistent diagrams developed by different people/groups must be merged into a cohesive whole.
• Sustainment. The relevant configuration items and connections between them will evolve over time according to the master schedules of the different projects. The E2E DFDs will must be maintained to remain correct, consistent, and useful. Their value will significantly decrease if they are not kept current, which often occurs with system-level projects that do not notify projects that are sustaining other systems within the SoS.
• Lack of tool support. Within the same diagram, most graphical modeling tools do not easily handle hardware, software, data flows, and aggregation from configuration items and line replaceable units up through systems to the SoS. Because of the large number of actions and interactions, the resulting diagrams can be hard to read when [re]drawn by tools that do not use placement and font size effectively
• Lack of standards. E2E DFDs are not an explicit part of DODAF, UML, SYSML, or AADL.
• Security classification. On real programs, the individual source diagrams and documents that form the basis for developing the E2E DFDs are typically marked For Official Use Only (FOUO). Some of the information may be classified, and even if all the source documents are unclassified, the combination of the information in the E2E DFDs may cause the completed diagrams to be classified. Classification must be considered and may make development, verification, and validation more difficult.

The preceding challenges make the initial E2E DFDs some of the most complex, difficult, and expensive architectural diagrams to create.

Lessons Learned

The E2E data flow analysis method was recently used on a US military program, where the following lessons were learned:

• Expect every SME interview to result in changes to the diagrams.
• Expect to conduct multiple interviews with key SMEs as your understanding improves and the diagrams evolve.
• Ensure free communication with SMEs by conducting interviews without attribution when discussing SME inputs with management.
• Show aggregation by nesting and color rather than stereotype labels. For example, nest software nodes inside the hardware nodes on which they execute. Nest these hardware nodes inside subsystem nodes and subsystem nodes inside system nodes. Then use arrows to connect the most nested nodes (e.g., software and network devices).
• Simplify the diagrams. Do not show node or arc redundancy or components smaller than configuration items, except perhaps some line replaceable units (LRUs).
• Because each SME and working group primarily has expertise in only certain parts of the SoS architecture, they should be interviewed separately. Trying to draw the E2E DFD diagram(s) in a large multi-group meeting would waste a great deal of the SMEs' valuable time. Hold off that group meeting until the diagrams are complete and relatively stable (i.e., until the presentation of results).

Because real diagrams may involve more than 50 path segments that need to be secured, security analysis should first concentrate on the highest risk interactions and system-internal interactions over which the program has control.

Wrapping Up and Looking Ahead

This series of blog posts has provided an overview of the E2E Mission Thread Data Flow Analysis (EMDA) method. The use of SoS-level mission thread data flow analysis provides many benefits, greatly increasing the understanding of how systems collaborate to implement mission among both acquisition organization and development organization personnel. Our aim with this work is to help individual architects and developers understand how the system components for which they are responsible fit into both their system and the overall SoS. Performing E2E mission thread data flow analysis is not trivial, however, and must be properly resourced if it is to paint a complete picture of how systems and their components interoperate.

Additional Resources

Read other posts by Don Firesmith.