Testing
Blog Posts
Introducing MLTE: A Systems Approach to Machine Learning Test and Evaluation
Machine learning systems are notoriously difficult to test. This post introduces Machine Learning Test and Evaluation (MLTE), a new process and tool to mitigate this problem and create safer, more …
Read More•By Alex Derr, Sebastián Echeverría, Katherine R. Maffey (AI Integration Center, U.S. Army), Grace Lewis
In Artificial Intelligence Engineering
The Myth of Machine Learning Non-Reproducibility and Randomness for Acquisitions and Testing, Evaluation, Verification, and Validation
A reproducibility challenge faces machine learning (ML) systems today. This post explores configurations that increase reproducibility and provides recommendations for these challenges.
Read More•By Andrew O. Mellinger, Daniel Justice, Marissa Connor, Shannon Gallagher, Tyler Brooks
In Artificial Intelligence Engineering
Redemption: A Prototype for Automated Repair of Static Analysis Alerts
This post introduces Redemption, an open source tool that uses automated code repair technology to repair static analysis alerts in C/C++ source code.
Read More•By David Svoboda
In Cybersecurity Engineering
Six Best Practices for Developer Testing
Code coverage represents the percent of certain elements of a software item that have been exercised during its testing....
Read More•By Robert V. Binder
Don't Play Developer Testing Roulette: How to Use Test Coverage
Suppose someone asked you to play Russian Roulette. Although your odds of surviving are 5 to 1 (83 percent), it is hard to imagine how anyone would take that risk....
Read More•By Robert V. Binder
Testing Concurrent Systems: Concurrency Defects, Testing Techniques, and Recommendations
Concurrency, which exists whenever multiple entities execute simultaneously, is a ubiquitous and an unavoidable fact of life in systems and software engineering....
Read More•By Donald Firesmith
In Cybersecurity Engineering
Enabling Shift-Left Testing from Small Teams to Large Systems
Shift left is a familiar exhortation to teams and organizations engaged in Agile and Lean software development. It most commonly refers to incorporating test practices....
Read More•By Nanette Brown
In Agile
Decision-Making Factors for Selecting Application Security Testing Tools
In the first post in this series, I presented 10 types of application security testing (AST) tools and discussed when and how to use them....
Read More•By Tom Scanlon
In Secure Development
10 Types of Application Security Testing Tools: When and How to Use Them
This blog post categorizes different types of application security testing tools and provides guidance on how and when to use each class of tool.
Read More•By Tom Scanlon
In CERT/CC Vulnerabilities
Seven Recommendations for Testing in a Non-Deterministic World
This blog posting describes a number of measures that have proved useful when testing non-deterministic systems.
Read More•By Donald Firesmith