Failure Analysis in CPS Simulink Models (Keynotes)

Failures observed in Simulink models are particularly hard to debug and explain, since any computation normally involves every, or most of, the elements in a model, and localization strategies based on the detection of the elements (only) activated by failed tests cannot work in this context. To address this challenge, approaches that explore the behavioural space of Simulink models to discover the internal behaviors that may characterize failing tests have been proposed. In particular, we recently worked on the generation and comparison of close passing and failing executions, to isolate the internal behaviors likely responsible for the failures. We investigated this approach both using models inferred from passing executions, which are then compared to failing executions [1–3], and by the straight comparison of pairs of passing and failing executions [4]. Experimenting these approaches require a large number of faults, which are seldom available in practice. In these cases, mutation testing is particularly helpful to run large experiments with synthetic faults. Unfortunately, the regular notion of mutant killing (i.e., the condition to reveal a fault) that requires generating a test that produces different outputs for the original and mutated version of a model, is not particularly useful in the context of CPS Simulink models. In fact, faults are normally trivial to activate and propagate to the output, thus being trivial to kill. Yet, a mutant-killing test might not be particularly useful, especially when a model must be validated against specific properties. In fact, the output difference generated by a mutant-killing test might not be enough to violate the available property, resulting in a test that would not expose the problem in a target model, even when it exercises the fault. To address this problem, we investigated the notion of property-based mutation testing, which requires the generation of tests that exercise faults, while magnifying their impact on the model up to causing the violation of the available properties [5, 6]. The talk will discuss recent advances obtained in failure analysis and fault injection in CPS Simulink models.