Juneberry is an open source Python tool, created by the SEI’s Artificial Intelligence (AI) Division, that verifies and validates machine-learning (ML) models by automatically training, evaluating, and comparing them against multiple data sets. The latest version of Juneberry adds the ability to simulate attacks that could disrupt computer vision systems.

Property inference attacks reveal information about an ML system’s training data. Using that information, attackers can craft and inject watermarks into image data to trick the ML system into unintended behavior.

Juneberry 0.5 allows users to prescribe ML model changes that simulate property inference attacks. “Very specialized semantics give users the knowledge of how these attacks are structured,” said principal engineer Andrew Mellinger. “These semantics are not available elsewhere.”

The attack simulation capability will allow ML developers and researchers to test their models for weaknesses against property inference attacks and ultimately protect ML-powered image classification systems.

Explore Juneberry at https://github.com/cmu-sei/juneberry.