Software Engineering Institute

This project, Portable High- Performance Inference at the Tactical Edge (PHITE), a collaboration with experts from the Department of Electrical and Computer Engineering at Carnegie Mellon University, applies performance engineering processes to the analysis of existing open source ML frameworks for embedded systems, to inform the development and optimization of a portable software library that can achieve significantly higher performance (10–100x power efficiency) for a set of ML applications across a range of targeted embedded devices.
 
Our approach proceeds in three phases:

1. selection and characterization of baseline applications and hardware platforms
2. software design and development
3. integration, evaluation, benchmarking, and demonstration of open source benchmark applications that have DoD relevance

In the first year of the project, we analyzed application workloads and performance modelling of the hardware, specified the first version of the API specification for the portable inference interface, and completed the initial implementations of a library of ML algorithms for each ultra-low power hardware platform. In year two, we will continue to optimize the portable library of ML algorithms (targeting 10x—100x performance gains over the baseline) for embedded devices and address any additional needs to support a live data stream.