Arguing Safety for AI-driven Systems

The first international AI safety standards, namely ISO/IEC TR 5469 and ISO/PAS 8800, have been released. The usage of AI in safety-critical settings offers new dimensions, including autonomous mobile robots, self-driving cars, perception systems etc. Traditional safety engineering primarily employs the V-model process, as a systematic means for translating requirements from specification to system hardware and software architecture and measuring their correct implementation during verification and validation. In contrast, AI-driven systems are predominantly data-driven and follow an iterative lifecycle. Within this lifecycle, the core objective is to compile an assurance argument that demonstrates the system's capability to implement necessary risk mitigations and address the inherent uncertainties of AI-based systems. The efficient assurance argument has been proposed as a means to characterize the key facets, necessary for system assessment, deployment readiness and certifiability. This study firstly, analyzes the first two international standards to determine their alignment with these characteristics and secondly identifies open questions, particularly the challenge of transitioning safety requirements assigned to AI-based safety-critical systems into quantifiable metrics.

Keywords: Assurance argument, functional safety, artificial intelligence, machine learning, standardization, certification, safety-critical system.