Integrating an Automated Action Recognition with a Simulation Approach to Analyze the Risks of the Newborn Life Support Algorithm

The Newborn Life Support (NLS) algorithm is an evidence-based protocol to resuscitate and stabilize a compromised baby. The resuscitation team should perform this in a timely and effective manner. Both technical and social aspects of this procedure can influence its outputs. Therefore, a comprehensive study integrating these aspects to analyze the performance of the NLS procedure is essential. Previous studies have investigated the significance of either the technical or non-technical aspects of the procedure using video recordings. However, this requires manual analysis of the recordings, making it lengthy and burdensome for clinicians. Therefore, an automated analysis of the recorded videos to identify variations in the NLS protocol would be highly beneficial. Our research introduces a potential solution to address these issues. We developed an integrated automated action recognition and simulation model that can be used to analyze the risks of the resuscitation procedure. The first element of the solution is the NLS simulation model developed using the Colored Petri Net approach. It considers both technical and non-technical aspects of the procedure, such as different types of respiratory devices, the levels of doctor’s experience, and the ability of the clinical staff to cope with the stressful situation during the procedure. The second element is the automated variation recognition system, which is built on a combination of image segmentation and action recognition techniques. The integration involves automated identification of the status of the wet towel removal step in the input NLS recordings, which is fed into the simulation model to analyze the risk of the clinical procedure in the long run. The risk is observed through the changes in the proportion of satisfactory conditions, resuscitation duration, and baby’s final heart rate. A simple graphical interface for this integrated system was also developed, allowing users to experiment with different NLS activity settings.

Keywords: Newborn Life Support, Colored Petri Nets, Action Recognition, Integrated Simulation Model, Clinical Risks, Healthcare Modelling.