Machine Learning
Blog Posts
DataOps: Towards More Reliable Machine Learning Systems
Decisions based on ML models can have significant consequences, and managing the raw material—data—in ML systems is a challenge. This post explains DataOps, an area that focuses on the management …
Read More•By Daniel DeCapria
In Artificial Intelligence Engineering
Evaluating LLMs for Text Summarization: An Introduction
Deploying LLMs without human supervision and evaluation can lead to significant errors. This post outlines the fundamentals of LLM evaluation for text summarization in high-stakes applications.
Read More•By Shannon Gallagher, Swati Rallapalli, Tyler Brooks
In Artificial Intelligence Engineering
Enhancing Machine Learning Assurance with Portend
This post introduces Portend, a new open source toolset that simulates data drift in machine learning models and identifies the proper metrics to detect drift in production environments.
Read More•By Jeffrey Hansen, Sebastián Echeverría, Lena Pons, Gabriel Moreno, Grace Lewis, Lihan Zhan
In Artificial Intelligence Engineering
Introducing MLTE: A Systems Approach to Machine Learning Test and Evaluation
Machine learning systems are notoriously difficult to test. This post introduces Machine Learning Test and Evaluation (MLTE), a new process and tool to mitigate this problem and create safer, more …
Read More•By Alex Derr, Sebastián Echeverría, Katherine R. Maffey (AI Integration Center, U.S. Army), Grace Lewis
In Artificial Intelligence Engineering
Cyber-Informed Machine Learning
This blog post proposes cyber-informed machine learning as a conceptual framework for emphasizing three types of explainability when ML is used for cybersecurity.
Read More•By Jeffrey Mellon, Clarence Worrell
In Cybersecurity Engineering
The Myth of Machine Learning Non-Reproducibility and Randomness for Acquisitions and Testing, Evaluation, Verification, and Validation
A reproducibility challenge faces machine learning (ML) systems today. This post explores configurations that increase reproducibility and provides recommendations for these challenges.
Read More•By Andrew O. Mellinger, Daniel Justice, Marissa Connor, Shannon Gallagher, Tyler Brooks
In Artificial Intelligence Engineering
The Top 10 Blog Posts of 2024
This post presents the top 10 most-visited posts of 2024, highlighting our work in software acquisition, artificial intelligence, large language models, secure coding, and more.
Read More•By Bill Scherlis
In Software Engineering Research and Development
Introduction to MLOps: Bridging Machine Learning and Operations
Machine learning operations (MLOps) has emerged as a critical discipline in artificial intelligence and data science. This post introduces MLOps and its applications.
Read More•By Daniel DeCapria
In Artificial Intelligence Engineering
Measuring AI Accuracy with the AI Robustness (AIR) Tool
Understanding your artificial intelligence (AI) system’s predictions can be challenging. In this post, SEI researchers discuss a new tool to help improve AI classifier performance.
Read More•By Michael D. Konrad, Nicholas Testa, Linda Parker Gates, Crisanne Nolan, David James Shepard, Julie B. Cohen, Andrew O. Mellinger, Suzanne Miller, Melissa Ludwick
In Artificial Intelligence Engineering
The Challenge of Adversarial Machine Learning
This SEI Blog post examines how machine learning systems can be subverted through adversarial machine learning, the motivations of adversaries, and what researchers are doing to mitigate their attacks.
Read More•By Matt Churilla, Nathan M. VanHoudnos, Robert W. Beveridge
In Artificial Intelligence Engineering
