Posted on by Systems Engineeringin
This post is the first in a series introducing our research into software and system complexity and its impact in avionics.
On July 6, 2013, an Asiana Airlines Boeing 777 airplane flying from Seoul, South Korea, crashed on final approach into San Francisco International airport. While 304 of the 307 passengers and crew members on board survived, almost 200 were injured (10 critically) and three young women died. The National Transportation Safety Board (NTSB) blamed the crash on the pilots, but also said "the complexity of the Boeing 777's auto throttle and auto flight director--two of the plane's key systems for controlling flight--contributed to the accident."
In a news report, acting NTSB chairman Christopher Hart stated that "The flight crew over-relied on automated systems that they did not fully understand." The NTSB report on the crash called for "reduced design complexity" and enhanced training on the airplane's autoflight system, among other remediations. Since complexity is a vague concept, it is important to determine exactly what it means in a particular setting. This blog post describes a research area that the Carnegie Mellon University Software Engineering Institute (SEI)is undertaking to address the complexity of aircraft systems and software.
The Growing Complexity of Aircraft Systems and Software
The growing complexity of aircraft systems and software may make it difficult to assess compliance to air worthiness standards and regulations. Systems are increasingly software-reliant and interconnected, making design, analysis, and evaluation harder than in the past. While new capabilities are welcome, they require more thorough validation and verification. Complexity could mean that design flaws or defects could lead to unsafe conditions that are undiscovered and unresolved.
In 2014, the Federal Aviation Administration (FAA) awarded the SEI a two-year assignment to investigate the nature of complexity, how it manifests in software-reliant systems such as avionics, how to measure it, and how to tell when too much complexity might lead to safety problems and assurance complications.
System Complexity Effects on Aircraft Safety
Our examination of the effects of system complexity on aircraft safety began in October 2014 and involved several phases, including an initial literature review of complexity in the context of aircraft safety. Our research is addressing several questions, including
Given answers to the metrics questions above, we will then identify which of our candidate metrics would best measure complexity in a way that predicts problems and provides insight into needed validation and certification steps. Other questions our research will address include:
The remainder of this post focuses on the findings of our literature review, which offered insights into the causes of complexity, the impacts of complexity, and three principles for mitigating complexity.
Causes of Complexity
While complexity is often blamed for problems, the term is usually not defined. When we performed a systematic literature search, we found this to be the case. As a result, our literature search broadened from simply collecting definitions to describing a taxonomy of issues and general observations associated with complexity. (This work was primarily performed by my colleague, Mike Konrad.)
Our literature review revealed that complexity is a state associated with causes that produce effects. We have a large taxonomy of different kinds of causes and another taxonomy of different kinds of effects. To prevent the impacts that complexity creates, one must reduce the causes of complexity, which typically include:
Impacts of Complexity
After a system is deemed complex--no matter the reason--it is important to examine the problems or benefits of that complexity. Many consequences of complexity are known and considered to be negative, including higher project cost, longer schedule, and lower system performance, as well as decreased productivity and adaptability.
Also, addressing critical quality attributes (e.g., safety versus performance and usability) in a system, or achieving a desired tradeoff between conflicting quality attributes, often results in additional design complexity. For example, to reduce the probability of a hardware failure causing an unsafe condition, redundant units are frequently designed into a system. The system then not only has two units instead of one, but it also has a switching mechanism between the two units and a way to tell whether each one is working. This functionality is often supported by software, which is now considerably more complex than when there was just one unit.
Complexity also impacts human planning, design, and troubleshooting activities in the following ways:
Three Principles for Mitigating Complexity
Our literature review also identified three general principles for mitigating complexity:
Of course, these principles overlap, and are expressed and sorted differently by different authors.
One could make the case that, in general, all practices of systems engineering started from a principle of managing complexity. The table below highlights some of the principles for managing complexity that we encountered during our literature review.
Our literature review also describes literature search results related to measurement and mitigation of complexity. Measurement and mitigation of complexity are the basis for the practice of good systems engineering, whether addressing collaboration among organizations and disciplines, requirements and systems abstractions and models, disciplined management and engineering, or even modular design, patterns, and refactoring.
The next blog post in this series will detail the second phase of our work on this project, which focuses on determining how the breadth of aspects of complexity can be measured in a way that makes sense to system and software development projects, and specifically for aircraft safety assurance and certification.
We welcome your feedback on our research.
To read the SEI report Reliability Improvement and Validation Framework, by Peter H. Feiler, John B. Goodenough, Arie Gurfinkel, Charles B. Weinstock, and Lutz Wrage, please click here.
The blog post is disseminated by the Carnegie Mellon University Software Engineering Institute in the interest of information exchange. While the research task is funded by the Federal Aviation Administration, the United States Government assumes no liability for the contents of this blog post or use thereof. The United States Government does not endorse products or manufacturers. Trade or manufacturer's names appear herein solely because they are considered essential to the objective of this blog post. The interpretations, findings, and conclusions in this report are those of the author(s) and do not necessarily represent the views of the funding agency. This document does not constitute FAA certification policy.