Report Advises Major AI Investments for U.S. Department of the Air Force
• Article
September 12, 2023—A new report recommends that the U.S. Department of the Air Force (DAF) invest in artificial intelligence (AI) development and prioritize AI testing and evaluation. The report Test and Evaluation Challenges in Artificial Intelligence-Enabled Systems for the Department of the Air Force, released last week by the National Academies of Sciences, Engineering, and Medicine, says DAF should commit to AI governance, workforce, research and development, digital infrastructure, testing and evaluation processes, experimentation, and tailored standards and practices.
Tom Longstaff, chief technology officer of the Software Engineering Institute (SEI), co-chaired the committee that wrote the report. “The future of AI-enabled systems in the DAF relies on significant advances in testing and evaluation, as well as a more tightly coupled feedback loop with Science and Technology, Acquisition, and Operations and Maintenance,” Longstaff said.
The Department of Defense (DoD) announced its AI strategy in 2019. In the few years since, the commercial and defense domains have seen an explosion of AI-enabled capabilities. To inform the DAF’s approach to AI, the DoD sponsored a committee, established by the National Academies of Sciences, Engineering, and Medicine, to hold a series of workshops on the Air Force’s ability to test and evaluate AI-enabled systems under operational conditions. The committee of industry, government, and academic experts met several times in 2022 and 2023 and released their findings on September 7.
The report notes that this early in the DAF’s incorporation of AI, the department lacks the capacity and digital infrastructure to support AI development, testing, and evaluation. The same is true for most organizations and agencies, but the DAF should make rapid adjustments. AI’s broad impact on the DAF will require the department to invest in various aspects of the technology, including workforce development, test and evaluation capabilities and protocols, data collection, human-system integration, adversarial testing, and trustworthiness.
“We found that a strong, empowered leader in DAF focused on how to verify the trustworthiness of AI-enabled systems is essential to deploying capable, trusted AI partners to our airmen,” said Longstaff.
As a federally funded research and development center, the SEI is well suited to advise DoD organizations on the latest digital technologies. The SEI has been on the forefront of helping the government acquire, build, and deliver AI systems that address mission needs. Its AI Division addresses the need for leap-ahead AI capabilities that are reliable, responsible, safe, fair, and transparent. The AI Division is leading a community-wide movement to mature AI engineering, an emergent discipline focused on developing tools, systems, and processes to enable the application of AI in real-world contexts.
Read the full report on the National Academies of Sciences, Engineering, and Medicine’s website. Learn more about the SEI’s AI research and development work.
