What We Know We Don't Know: Systematic Risk Identification from Contradictory Safety Documentation

Ensuring accountability and responsibility in human-centered safety-critical systems, such as autonomous vehicles and medical devices, is essential. These systems must assure safety, comply with normative constraints, ethical principles, and regulatory standards while navigating complex human-machine interactions. Existing methods fall short in verifying normative properties and attributing responsibility for many reasons. Insufficient knowledge of the unknown risks that remain in systems being one of them. We propose a Kripke-based approach that formally connects the Open World Assumption of ontological Knowledge Graph with Closed World Assumption analysis for systematic risk identification. The approach presented can increase knowledge of the unidentified risks in the set of the closed world assumptions of the total residual risk. Our methodology addresses the challenge that Knowledge Graph extraction from informal documentation is not formulated precisely enough to be validated or verified, preventing automation and correctness guarantees. We construct Kripke structures where each state represents a world model satisfying Knowledge Graph configurations derived from contradictory documentation. The framework can integrate expert feedback, ensuring alignment with domain knowledge and regulatory expectations. A case study on a simplified automotive seatbelt warning system demonstrates the framework's ability to compare documented safety properties against latent hazard criteria and reveals a 40.9 % coverage gap: states classified as safe by specification but still harboring residual risks due to sensor-actuator conflation and unobservable safety properties.

Keywords: Unknown risks identification, Open-World Assumption, Known Unknowns, safety-critical systems, accountability and responsibility.