Despite the recent advances in automatic test generation, testers must still write test oracles manually. If formal specifications are available, it might be possible to use decision procedures derived from those specifications. We present a technique that is based on a form of specification but also leverages more information from the system under test. We assume that the system under test is somewhat redundant, in the sense that some operations are designed to behave like others but their executions are different. Our experience in this and previous work indicates that this redundancy exists and is easily documented. We then generate oracles by cross-checking the execution of a test with the same test in which we replace some operations with redundant ones. We develop this notion of cross-checking oracles into a generic technique to automatically insert oracles into unit tests. An experimental evaluation shows that cross-checking oracles, used in combination with automatic test generation techniques, can be very effective in revealing faults, and that they can even improve good hand-written test suites.

Carzaniga, A., Goffi, A., Gorla, A., Mattavelli, A., Pezze', M. (2014). Cross-checking oracles from intrinsic software redundancy. In Proceedings of the 36th International Conference on Software Engineering (pp.931-942) [10.1145/2568225.2568287].

Cross-checking oracles from intrinsic software redundancy

PEZZE', MAURO
2014

Abstract

Despite the recent advances in automatic test generation, testers must still write test oracles manually. If formal specifications are available, it might be possible to use decision procedures derived from those specifications. We present a technique that is based on a form of specification but also leverages more information from the system under test. We assume that the system under test is somewhat redundant, in the sense that some operations are designed to behave like others but their executions are different. Our experience in this and previous work indicates that this redundancy exists and is easily documented. We then generate oracles by cross-checking the execution of a test with the same test in which we replace some operations with redundant ones. We develop this notion of cross-checking oracles into a generic technique to automatically insert oracles into unit tests. An experimental evaluation shows that cross-checking oracles, used in combination with automatic test generation techniques, can be very effective in revealing faults, and that they can even improve good hand-written test suites.
paper
testing oracles, intrinsic software redundancy
English
International Conference on Software Engineering
2014
Proceedings of the 36th International Conference on Software Engineering
978-1-4503-2756-5
2014
931
942
none
Carzaniga, A., Goffi, A., Gorla, A., Mattavelli, A., Pezze', M. (2014). Cross-checking oracles from intrinsic software redundancy. In Proceedings of the 36th International Conference on Software Engineering (pp.931-942) [10.1145/2568225.2568287].
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/10281/172080
Citazioni
  • Scopus 41
  • ???jsp.display-item.citation.isi??? 24
Social impact