Overview
For well-trained ML models deployed in safety-critical autonomous systems, failures are rare but potentially catastrophic. Exhaustive search or random testing is prohibitively expensive. This work addresses the challenge of finding these failure scenarios faster than traditional randomized search.
Approach
The central idea is to separate the input data space into regions of high failure probability and regions of low/minimal failure probability. This separation is informed by:
- Training data observations
- Data drawn from real-world statistics
- Domain knowledge about the system’s operating conditions
By focusing search efforts on high-probability failure regions, we achieve significantly faster discovery of safety-critical edge cases compared to uniform random sampling.
Relevance to Assured Autonomy
This work directly addresses the DARPA Assured Autonomy program’s core challenge: verifying correctness of learned components in autonomous systems. By efficiently finding failure modes, we enable designers to retrain and improve models before deployment, contributing to the formal safety assurance pipeline.