Software systems may fail in production environments, causing system crashes, erroneous outputs, and overall system instability. Thoroughly testing software systems in development environments can reduce but not avoid failures, due to both the complexity of software applications, which may lead to a myriad of execution conditions impossible to sample exhaustively, and the many behaviors that emerge in production, which can be hardly predicted and exercised during development. This paper presents field-ready test cases, tests designed to run in production environments, aiming to proactively execute soft-ware components in yet unexplored execution scenarios, exposing error states before they result in system failures. Intuitively, the approach conceives the production environment as a testbed for opportunistically executing unit test cases that exploit the objects that become available as test data. The paper presents the results of a set of experiments with field-ready test cases that we produced for the JFreeChart and Apache Commons Lang libraries. Our field-ready test suites execute 64% of the faults that the original test suites miss, and exposes 33% of the missed faults.

Gazzola, L., Mariani, L., Orru, M., Pezze, M., Tappler, M. (2022). Testing Software in Production Environments with Data from the Field. In Proceedings - 2022 IEEE 15th International Conference on Software Testing, Verification and Validation, ICST 2022 (pp.58-69). Institute of Electrical and Electronics Engineers Inc. [10.1109/ICST53961.2022.00017].

Testing Software in Production Environments with Data from the Field

Gazzola L.;Mariani L.;Pezze M.;
2022

Abstract

Software systems may fail in production environments, causing system crashes, erroneous outputs, and overall system instability. Thoroughly testing software systems in development environments can reduce but not avoid failures, due to both the complexity of software applications, which may lead to a myriad of execution conditions impossible to sample exhaustively, and the many behaviors that emerge in production, which can be hardly predicted and exercised during development. This paper presents field-ready test cases, tests designed to run in production environments, aiming to proactively execute soft-ware components in yet unexplored execution scenarios, exposing error states before they result in system failures. Intuitively, the approach conceives the production environment as a testbed for opportunistically executing unit test cases that exploit the objects that become available as test data. The paper presents the results of a set of experiments with field-ready test cases that we produced for the JFreeChart and Apache Commons Lang libraries. Our field-ready test suites execute 64% of the faults that the original test suites miss, and exposes 33% of the missed faults.
paper
Fault detection; Field testing; Field-Ready test cases;
English
15th IEEE International Conference on Software Testing, Verification and Validation, ICST 2022 - 4 April 2022 through 13 April 2022
2022
Proceedings - 2022 IEEE 15th International Conference on Software Testing, Verification and Validation, ICST 2022
9781665466790
2022
58
69
none
Gazzola, L., Mariani, L., Orru, M., Pezze, M., Tappler, M. (2022). Testing Software in Production Environments with Data from the Field. In Proceedings - 2022 IEEE 15th International Conference on Software Testing, Verification and Validation, ICST 2022 (pp.58-69). Institute of Electrical and Electronics Engineers Inc. [10.1109/ICST53961.2022.00017].
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/10281/395074
Citazioni
  • Scopus 2
  • ???jsp.display-item.citation.isi??? 1
Social impact