Systems engineers working today face many challenges, both in building the complex systems of systems of the future and in building the complex systems of which they are composed. Systems engineers need to be able to design around stable requirements when there are long-lead manufactured items required, and they also need to evolve the design along with changing requirements for larger systems. Software plays an integral role in helping systems engineers accomplish these goals.

The importance of software engineering to systems engineering, and vice-versa, cannot be overstated. As I stated in an earlier blog post

Systems engineers are responsible for ensuring that every part of the system works with every other, and because software represents the intelligence of human-made systems, it is especially important to the integration of the system. Yet many chief systems engineers and program managers have had more experience in mechanical or electrical engineering than in software or software engineering.

In this post, I examine whether the ways in which systems engineers talk about software has changed in the two decades by examining INCOSE's Systems Engineering journal. The reason for this inquiry is to understand whether systems engineers have come to accept that software is a huge and growing fraction of the systems they engineer and to understand how to partner with them on programs and in acquisition efforts.

I have been a member of the International Council on Systems Engineering (INCOSE) for 25 years and an associate editor of INCOSE's Systems Engineering journal for 15 years. In commemoration of the 20-year anniversary of the journal, I recently analyzed the publication to determine whether systems engineering has evolved in how it views software. Specifically, I examined whether the ways in which systems engineers talk about software have changed during that time. This post, which is excerpted from a recently published paper, summarizes the article for the benefit of software engineers; see the published paper for additional information.

Foundation

The systems whose development programs employ systems engineers are increasingly dependent on and controlled by software. For example, the effort to write and maintain software on fighter aircraft now exceeds all other modernization costs. As systems become increasingly software dependent, the relationship and understanding between systems engineers and software engineers becomes increasingly important.

For this analysis, I examined all the articles from two years in Systems Engineering, Volume 3 (2000) and Volume 18 (2015), to determine whether the manner in which systems engineers talk about software has changed in that 15 years.

By reviewing articles published in those years, I specifically sought to answer these three research questions:

RQ1. How has systems engineering's treatment of (understanding of, actions regarding) software changed over the 20 years of the journal?

RQ2. Has systems engineering grown toward more knowledge of and respect for software, use of software, and integration with software, in the past 20 years?

and

RQ3. Has systems engineering become less waterfall-driven, process-oriented, and heavyweight, and more agile and model-based, in the same timeframe?

I chose the third year (Volume 3, 2000) for three reasons:

1. It avoided the first year, when the journal was becoming established.
2. It avoided the second year, which included a specific issue focused on model-based systems engineering, which might skew the results of the question about how systems engineering treatment of software has changed.
3. It was the middle year of the first five years.

I chose the 15th year (Volume 18) for two reasons:

1. It is the middle of the last 5 years (symmetric with the early chosen year).
2. It was the last year for which I had access to hardcopy journals, which made the analysis easier.

In the results of my analysis, I included the rates at which a certain topic appeared out of either the 20 articles in Volume 3 or 29 articles in Volume 18.

Note that in the following discussions, many articles are positive "hits" for multiple topics and thus appear in discussions multiple times. For example, an article about assessing risk by Haimes et al., published in 2015, touches upon issues of economics, systems of systems (SoSs), and security.

Analysis method

In reading and analyzing each article, I examined the kinds of topics each article addressed. I created a list of 85 topics, starting with the topics implied by the research questions and then added more as I read articles about them. I subsequently ensured that I had checked each article for whether it addressed each topic.

The list included a section of theory or practice topics (Items included theory, big-system practice, project management, processes, etc.); a SysE/software topics section (SysE in general; SysE, but implicitly software--as in smart systems; software intensive systems, etc.), the lifecycle phase section (proposals, contract negotiation, requirements, etc.), and sections on models and simulation, tools, domain (industries), and management activities (metrics, cost, risk, etc.).

Results of My Analysis

After analyzing the difference in subjects between the two years, I noted a number of differences in the frequency of topics between volumes 3 and 18:

Did not discuss any software at all
7/20 in 2000 (35 percent) vs. 12/29 in 2015 (41 percent)

None of the articles in either year discussed software engineering in general. Rarely, an article would mention software in a specific way as part of a combined system-software program. In general, systems discussions stayed above the level of software requirements or software-specific issues, focusing instead on system-level capabilities, even if those capabilities were provided by software. This is a bit unfortunate as it fails to give credit to those making the hard part of the system work, but it has precedents in other areas. For example, NASA talks about "a NASA spacecraft" (the spacecraft they contracted for and are operating) without mentioning the contractor that built it.

Discussed complex systems, socio-technical systems, or systems of systems (SoS)
1/20 in 2000 (5 percent) vs. 8/29 in 2015 (28 percent)

Larger systems were more prevalent a topic in Volume 18 than Volume 3. The only article from 2000 [Beckerman, 2000] that met this criterion talked in a very general way about complex systems science, chaos, emergence and holism. In 2015 four articles addressed the complex nature of the system explicitly: one [Haimes, 2015a] focused on emergent complex SoS; another [Davendralingam, 2015] examined SoS architectures; a third [Haimes, 2015b] addressed complex interconnected SoS; and the fourth [Kilicay-Ergin, 2015] addressed SoS acquisition.

Other articles applied a new algorithm or theory to a specific system. One article [Davison, 2015] looked at an in-space transportation infrastructure while another [Luo, 2015] examined an array of space-based infrared sensors; a third [Rosen, 2015] examined wide area surveillance/screening; and a fourth [Al Kawasmi, 2015] looked at the global carbon trading infrastructure. The fact that these four authors did not highlight the SoS or complex system nature of the systems they addressed underscores that such systems have become standard, accepted characteristics of today's systems engineering efforts.

Discussed systems where software is not considered a separate part, rather the means by which capability is achieved
1/20 in 2000 (5 percent) vs. 8/29 in 2015 (28 percent)

I added this category very late in the review process after looking for specific treatment of software in the 2015 journal articles. There seemed to be quite a few articles that did not mention software at all, but the authors were clearly looking at systems that contained it. In fact, the software that the systems contained is what primarily created the required capability. Five of those systems are mentioned in the previous paragraph ([Haimes, 2015a], [Davison, 2015], [Rosen, 2015], [Luo, 2015], and [Kilicay-Ergin, 2015]); others include an operational system to determine where the DoD would place contingency bases [Nottage, 2015], wind farm control [Trantham, 2015], and six systems in different industries, ranging from a machine that places fibers for composites to a vehicle localization system for an unmanned ground vehicle [Engel, 2015].

The one earlier-year article that met this category [MacLeod, 2000], considered explicitly, "Man-Machine Systems" in space and their human factors. This article's terminology feels outdated.

Historically, the U.S. has thought of defense systems as vehicles or other hardware items that are the solutions to defense requirements or needs. It is clear that the articles from 2015 see systems as much larger than any unique hardware unit, essentially encompassing large and complex systems-of-interacting-systems. This is as it should be, and it is particularly noteworthy that the proportion of aerospace and defense-domain articles is higher in the later volume (13/29 or 45 percent) than in the earlier volume (4/20 or 20 percent), so this trend cannot be explained away by a change in domain.

Discussed decision-making and/or trade studies
2/20 in 2000 (10 percent) vs. 8/29 in 2015 (28 percent)

There were considerably more articles in 2015 that addressed decision making and trade studies directly than in 2000. The two articles published in 2000 only addressed decision making and trade studies as one of many systems engineering activities: one [Martin, 2000], as a process in the ANSI/EIA 632 standard, and the other, [Osmundson, 2000], as a part of systems engineering for information systems.

In 2015, the articles that mentioned decision-making or trade studies were more focused. One [Haimes, 2015a] improved the conventional linear trade study methodology by also addressing Pareto-Optimal decisions. Another [Nottage, 2015] showed how model-based systems engineering can improve operational decisions.

Several articles published in 2015 applied trade studies to a system: One [Connelly, 2015] showed how to choose the best technology maturation investments for an uncertain future. Another [Zhang, 2015] applied a systemic methodology to selecting alternative designs for power plants. A third [Cardin, 2015] addressed decision methodologies to select which capital-intensive improvements to fund in liquefied natural gas technologies. A fourth article [Davendralingam, 2015] showed how to evolve a systems of systems architecture using scenarios, and finally, a fifth article [Kilicay-Ergin, 2015] showed how to select system incentives that will lead to the best SoS outcomes, using options theory.

While the topic of trade studies does not directly address the question of software within systems engineering, it is useful to those trying to introduce systems engineering to software organizations. This is because one of the historical ways in which systems engineering does something more reliably than software is in formalizing (and performing informal and formal) trade studies. The 2015 articles all address software-intensive systems or at least use software tools within systems engineering, tying the two disciplines more tightly together than before.

Discussed Security
0/20 (0 percent) vs. 4/29 (13.7 percent)

There was no discussion of security at all in 2000. In 2015 four articles looked into security (a particularly important quality attribute); in all cases this was cybersecurity rather than physical security. One [Al Kawasmi, 2015] considered the Bitcoin business model in part because of its answer to security issues, and two others [Ali 2015] and [Haimes, 2015b] addressed economic fallout of cyber incidents. A fourth [Trantham, 2015] looked specifically at cybersecurity for wind farms.

Discussed quality attributes other than security
1/20 in 2000 (5 percent) vs. 7/29 in 2015 (24.1 percent)

In 2000 only one article addressed quality attributes (other than security), in this case changeability: this article [Fricke, 2000] looked at adaptability to expected changing requirements.

In contrast, in 2015, six of the 29 articles addressed one of the following quality attributes: privacy [Al Kawasmi, 2015], flexibility ([Zhang, 2015] and [Cardin, 2015]), modularity [Suh, 2015], adaptability ([Engel, 2015] and [Mirshekarian, 2015]) and evolvability [Mirshekarian, 2015]. The seventh [Kilicay-Ergin, 2015s] addressed "utility" in general, which is equivalent here to quality attributes.

Based on my analysis of two years of articles published in the Systems Engineering journal, there is more interest today in the larger system (e.g. economics, simulation of the environment, systems-of-systems), on quality attributes (also called ilities) and on modeling, management, and technology maturity. There is less interest today in process, requirements, and oddly, human factors. All but the last can be explained by systems engineering having moved away from initially deeply-explored topics into new areas. Human factors remains important, yet the topic not addressed as extensively.

External validation

To see whether the selected years were representative of the whole first five years, and whether the articles in the journal Systems Engineering represented the field of systems engineering as a whole, I performed a lexicographic trend analysis. Using two online databases that review articles of interest to systems and software engineering Compendex and Inspec), I verified the frequency of usage of terms during the entire lifetime of the journal. Some of the conclusions evident from this analysis are as follows

• mentions of systems of systems increased by almost eight times during this time period; similarly mention of both the term system and any one of these three grew: infrastructure, energy, or health. This is consistent with the analysis of articles showing an increase in mentions of larger and complex systems.
• the term decision grew steadily over time, showing 59 percent more mentions in 2015 than in 2000, which is in the same direction as the growth shown in the two years of articles in the journal, but not as high.
• embedded systems grew much more than information systems over the 15-year period, which is consistent with an increase in interest of the relationship of software to hardware in complex systems

Limitations

This article looked at various software-related topics that were addressed in articles in two volumes of a systems engineering journal. It would be interesting to study the appearance of systems engineering concepts or even the term systems engineering in software-focused journals, to see whether a reciprocal interest can be seen to grow over that time period.

Future Work

Since most systems engineering articles do not discuss software, the first effort needed is to ensure that systems engineering adequately considers the challenges of software on today's systems. The International Council on Systems Engineering (INCOSE) has begun a working group on this topic. Software must also be more cognizant of the purpose and role of systems engineering, which is different from software engineering or software architecting in its understanding of hardware engineering, its breadth of knowledge, and its focus on long-term viability of the entire system to its customers.

The future of systems and software engineering should also include identifying how to react when the nature of system components changes, to ensure safety, among other quality attributes. For example, some systems that have traditionally been custom made and systems-engineered are being replaced with commercial off-the-shelf (COTS) systems that were not made for a military purpose. The new U.S. Navy submarine U.S.S. Colorado has replaced clunky, heavy, $38,000 periscope controllers with $20 Xbox 360 controllers. Given that such controllers were never intended for military operation, are they effective enough in expected battle situations, and are they cyber-secure enough?

In another example, the FAA had to issue a permanent rule prohibiting airline pilots from using any personal electronic devices during operation. Among other incidents that prompted the rule was one in which four people were killed in Missouri when a medical helicopter ran out of fuel. The NTSB report on the accident stated that the pilot's "distracted attention due to personal texting during safety-critical ground and flight operations" was a contributing factor to the accident. It will be a challenge to manage the use of COTS devices when they are relatively plentiful and easy-to-use but may provide dangers in safety-critical contexts. Given that custom-built systems are using more COTS components research should

• Determine whether the COTS components and the systems using them already have had some version of systems engineering and if not, whether any problems result
• Identify improvements to how systems engineering is done today, and what should be done with COTS systems, to resolve the problems, and finally
• Identify paths and business cases to achieving the improvements

Additional Resources

Read the paper Adapting Systems Engineering for Software-Intensive Systems, which I presented at the INCOSE 2004 - 14th Annual International Symposium Proceedings.

Read the 2018 INCOSE paper, INCOSE Working Group Addresses System and Software Interfaces.