Software Engineering Institute

December 18, 2024—Machine learning (ML) models are powerful but brittle components of intelligent software systems. Developers strive to select model training data that is as close as possible to the deployment environment. If the input data from the deployment environment drifts away from the training data, the model can produce incorrect outputs, endangering the system’s mission. The SEI recently released an open source toolset called Portend that model developers can use to simulate ML data drift, set guardrails, and give an alert when model data drifts too far, enhancing ML assurance.

Imagine an autonomous unmanned aerial vehicle (UAV) using ground imaging to navigate over a flood-damaged city. The UAV’s ML model takes in new image data, but it is so different from the model’s pre-flood training data that the UAV can no longer navigate to its destination.

Data drift like this is a problem for ML assurance, or ensuring that the model performs as expected, because it is hard to know when it has occurred. Correcting drift requires model retraining, an expensive and resource-intensive process. Most model developers either retrain the model periodically, potentially wasting resources if drift is not present, or after the model’s behavior has already failed in deployment. If the mission is urgent or there is no relevant data, retraining may not even be an option.

SEI researchers created the Portend toolset to enhance ML assurance. While drift detection algorithms exist, Portend is an end-to-end solution that simulates a model’s drift, sets drift thresholds, and alerts system users when the model exceeds them.

Portend uses libraries of known types of data drift to induce artificial drift in an ML model. “Then you can see how models react to that artificial drift,” said Jeffery Hansen, a senior ML research scientist at the SEI and principal investigator on the Portend project. “Now you have ground truth on what data is and isn’t drifted, and you can see the difference in the model’s behavior.” Users can then configure a Portend monitor to raise an alert if it sees that behavior in operation, signaling drift.

The UAV navigating the flooded city, for example, would get an alert that the model’s output was no longer reliable due to differences between the terrain data and the training data. The system or its operators could then take corrective actions, such as continuing on, finding alternate routes, or aborting the mission. Later, developers could examine that operational data and decide whether to retrain the model.

Image recognition and navigation are just two of many possible applications for Portend, which is agnostic to model and data type. This adaptability was built into Portend, said Sebastián Echeverría, an SEI senior software engineer. “Our project provides some sample libraries for drift generation, but it’s very flexible and easy to add different libraries and methods for different contexts and environments.”

Better ML assurance helps all the stakeholders of ML-enabled missions: operators are made immediately aware of drift’s effect on the mission, developers know when to retrain the model, and decision makers have more data on how to budget ML training resources.

The SEI invites ML developers to try Portend, available on the institute’s GitHub site, and provide feedback through GitHub issues or by emailing info@sei.cmu.edu. Learn more about Portend in Hansen’s 2023 SEI Research Review presentation.