Annotating Maintenance Reports with a Domain-Specific Ontology and a Large Language Models to Extract Reliability Data from Industrial Systems

While maintenance reports contain valuable data for supporting reliability analysis, extracting them is hindered by technical jargon, extreme conciseness and lack of standardization. To address these challenges, we propose a method that combines a Large Language Model (LLM) with a domain-specific ontology to automatically annotate maintenance reports. Specifically, the annotation task involves assigning semantic labels to specific text spans and identifying relations between them. The identified entities and relations subsequently serve for deriving quantitative reliability metrics, such as component failure rates. The proposed method is validated using a repository of maintenance reports concerning malfunctions in traction systems of a fleet of freight transport trains. The obtained results show that the method can effectively annotate maintenance reports, achieving an average F1-scores of 0.903 and 0.733 for entities and relations, respectively.

Keywords: Natural Language Processing, Large Language Model, Annotation, Ontology, Maintenance Reports.