The world we live in is increasingly digital, synthetic, and fueled by data. The software it is built on is developed with such speed and automation that we must think about security in a new way. And in today's age of artificial intelligence (AI), cyber adversaries operate with speed and dexterity in a world of ever-changing attack surfaces.

In light of this constantly evolving cyber landscape, our researchers work to secure our infrastructure and resources and gain a cyber advantage over our adversaries. As this blog post will detail, this challenge requires that we transcend the capabilities of our adversaries by developing systems in agile, adaptive environments that are resilient in both design and operation. At a time when opaqueness pervades our world, we must operate with transparency and awareness. The SEI's focus on the interrelated issues of software engineering, cyber, and artificial intelligence allows us to address present and future threat environments. Our position to address these issues is enhanced through

• our collaborative research with Carnegie Mellon University faculty
• our direct engagement with the Department of Defense (DoD) and the defense industrial base
• our long-standing partnerships with the public and private sectors
• access to operational cyber networks, systems, and data
• the ability to inform and collaborate with DoD policy makers

­A Landscape of Disinformation and Poisoned Data

The SEI's CERT Division has grown from a handful of analysts who gathered in 1988 to address the Morris worm and its successors to hundreds of dedicated researchers, principally located in Pittsburgh and Arlington, working with a community of partners across the globe. Today the CERT Division performs leading-edge research both pushing the state of the art and closing the gap between the state of the art and the state of the practice.

Attackers today traffic in disinformation and "poisoned" data that impede the integrity of system function. Researchers have noted that attackers now exploit vulnerabilities in the very data sets used to train machine learning systems, with the aim of ensuring that their malicious code evades detection and leaving cyber defenders with little to no knowledge of the attackers. Our focus is on helping cyber defenders understand their roles, their enemy, security controls, and system risk in this new, increasingly complex environment.

A New Approach for a New Landscape: Verifiable Confidence

Our approach focuses on increasing confidence in the technology and its use throughout the missions of DoD and establishing the means to verify trust under the contested environments that we operate in. This includes advancing methods in testing, security controls, and systemic risk, all of which have to be understood and implemented with increased formalism, greater automation, and a stronger link across the design, development, and operations lifecycle. We prioritize not only the adoption of trustworthy components, including the adoption of state-of-the-art capabilities, but also the development of confidence in the tactics, techniques, and procedures that undergird operations in cyberspace. We will also continue to prioritize efforts to include security in system architecture and design activities. We work across the institute to help government agencies shift left, or increase focus on security earlier in the lifecycle, by incorporating Agile and DevOps practices into software and systems development. Each of these objectives becomes even more important in a world fueled by the data and autonomy in operations today and will become even more important over the next five years.

A National Security Priority

Amid these developments across the cyber realm, our nation's leaders have made U.S. dominance in cyberspace a national security priority, which informs our focus on attaining autonomous cyber operations and resilience in DoD missions by

• delivering and sustaining trustworthy and resilient data, systems, and enterprises and assuring supply chains. Our work focuses on ensuring data veracity and the development of security-instrumented lifecycles, model-based security and resiliency engineering, automated code analysis and repair, and supply-chain risk management.
  
  
• ensuring that cyber operators are capable of responding to near-peer adversaries regardless of the environments. To prepare the cyber workforce to counter synthetic adversaries, the SEI develops and deploys "train-as-you-fight" models, simulations, platforms, and exercises as well as tools that ensure measurable confidence in readiness for cyber missions.
  
  On a separate front, we are also focused on threat modeling for preemptive risk and for identification and mitigation.
  
  
• enabling rapid adoption and secure integration of promising technologies for autonomous cyber operators. We pursue automation and autonomy in key cyber tradecraft areas with efficiency and speed by focusing on malware analysis, cyber forensics, situational awareness, insider-threat detection and mitigation, and incident response capabilities.
  
  We also work to advance the trusted use of AI-enabled and autonomous capabilities and increase the robustness of AI-enabled solutions. More specifically, we examine the development of metrics and tools for verification and validation of AI, focusing on correctness, robustness, and scalability as well as understanding and mitigating risks from unique attack vectors of AI-enabled systems.

• preparing the cyber workforce. We need trained professionals who can operate in this new environment. A global cyber workforce shortage exacerbates this challenge--3 million cybersecurity professionals worldwide and nearly 500,000 in North America--an issue that has been cited as a top priority by the White House and other federal agencies. The SEI supports a range of training activities to bolster and develop the cyber workforce. In partnership with Carnegie Mellon University's Heinz College, we have developed C-suite training, including our Chief Information Security Officer (CISO) certificate and our Chief Risk Officer (CRO) certificate programs. Earlier this year, we released a suite of tools to help developers of cybersecurity training create realistic simulations and training exercises for trainees who operate in closed (often classified) environments with no ability to connect to the Internet.

The Role of AI

Machine learning and AI help us deploy algorithms to tackle these challenges at speed and at scale. Across the SEI, we are using machine learning to change the process of software development and improve its security. AI helps defenders work at the speed and scale of our adversaries while enhancing our ability to focus on data provenance, mission execution, and complexity.

AI-enabled automation has tremendously improved defenders' effectiveness and efficiency, perhaps even tipping the scales in the defenders' favor. Within the CERT Division, we are focused on bringing AI technologies into security missions to help them execute more effectively.

Though AI carries much promise, it has also increased the breadth of attack surfaces for our adversaries. "Adversarial AI attacks succeed, in most cases, by predicting the decisions machine learning models will make and then manipulating subsequent sets of data to produce the attacker's desired outcomes--rather than the correct decisions," Accenture stated in a recent report. Our researchers are focused on helping defenders understand how adversarial use of AI is going to be leveraged and helping our nation's defenders understand if they are fighting a machine or a human.

Underlying all of our work is the idea that AI is written in software, which makes it inherently vulnerable to attack and exploits.

Looking Ahead

In our increasingly complex world that is fueled by data and powered by technology, our focus is on helping the DoD and other federal agencies maintain a cyber advantage. To accomplish this goal, we are taking the long view, stepping back and left to ensure that systems are developed and acquired with security at the forefront. While our approach will continue to evolve, we remain committed to our mission of finding the government answers that will scale in today's environment and easily adapt to future challenges.

Additional Resources

Learn more about our work in the 2018 SEI Year in Review.

View the SEI Podcast A Technical Strategy for Cybersecurity with Bobbie Stempfley.

View the SEI Podcast Leading in the Age of Artificial Intelligence with Thomas Longstaff, SEI CTO.

Learn more about some recent projects in AI at the SEI, 2019.

Get some practical steps for implementing AI with cyber intelligence in our Artificial Intelligence and Cyber Intelligence: An Implementation Guide, 2019.