Towards automated Markov modeling and simulation of black-box systems

Ever-increasing operational data from interconnected smart technical systems create opportunities for data-driven modelling. Yet inferring reliable system state dynamics from system level data alone, without knowledge of internal structure, functional design, or subsystem models, remains a fundamental challenge in black-box modelling. While the existing literature addresses specific aspects of this challenge, a unified methodology integrating these interdependent tasks into a closed-loop, criterion-driven workflow with automated convergence is absent. To address this gap, the present paper presents a seven-step workflow that systematically integrates scattered approaches into a closed-loop, iteratively converging process. The workflow covers: First, define system state, subsystem state and transition model-space based on system design assumptions. Second, generation of system state time histories. Third, disturbance and measurement modeling are incorporated to represent noise, partial observability, and environmental effects explicitly. Fourth, model structure and parameter learning include Bayesian inference via Markov chain Monte Carlo (MCMC) simulation. Fifth, hidden state estimation and forecasting. Sixth, evaluation and uncertainty are assessed to know the quality of model. Seventh, criterion-driven deployment monitoring with automated iteration until convergence. Each workflow step is supported by reviewed literature, clarifying methods, dependencies between steps and research gaps. Comparison with five existing approaches confirms that no prior work unifies structure selection, parameter learning, convergence criteria, and feedback routing for black-box reliability systems. Key research gaps are identified in automated state space construction. The method provides structured guidance for practitioners modeling partially observable systems, emphasizing transparency, reproducibility, and systematic iteration for black-box system under non-stationary conditions.

Keywords: Black-box state modelling, Transition parametrization, MCMC simulation, Automated convergence, System health monitoring, Uncertainty quantification.