Search-based software testing (SBST) typically relies on fitness functions to guide the search exploration toward software failures. There are two main techniques to define fitness functions: (a) automated fitness function computation from the specification of the system requirements, and (b) manual fitness function design. Both techniques have advantages. The former uses information from the system requirements to guide the search toward portions of the input domain more likely to contain failures. The latter uses the engineers’ domain knowledge. We propose ATheNA, a novel SBST framework that combines fitness functions automatically generated from requirements specifications and those manually defined by engineers. We design and implement ATheNA-S, an instance of ATheNA that targets Simulink® models. We evaluate ATheNA-S by considering a large set of models from different domains. Our results show that ATheNA-S generates more failure-revealing test cases than existing baseline tools and that the difference between the runtime performance of ATheNA-S and the baseline tools is not statistically significant. We also assess whether ATheNA-S could generate failure-revealing test cases when applied to two representative case studies: one from the automotive domain and one from the medical domain. Our results show that ATheNA-S successfully revealed a requirement violation in our case studies.

(2024). Search-Based Software Testing Driven by Automatically Generated and Manually Defined Fitness Functions [journal article - articolo]. In ACM TRANSACTIONS ON SOFTWARE ENGINEERING AND METHODOLOGY. Retrieved from https://hdl.handle.net/10446/262251

Search-Based Software Testing Driven by Automatically Generated and Manually Defined Fitness Functions

Menghi, Claudio
2024-01-01

Abstract

Search-based software testing (SBST) typically relies on fitness functions to guide the search exploration toward software failures. There are two main techniques to define fitness functions: (a) automated fitness function computation from the specification of the system requirements, and (b) manual fitness function design. Both techniques have advantages. The former uses information from the system requirements to guide the search toward portions of the input domain more likely to contain failures. The latter uses the engineers’ domain knowledge. We propose ATheNA, a novel SBST framework that combines fitness functions automatically generated from requirements specifications and those manually defined by engineers. We design and implement ATheNA-S, an instance of ATheNA that targets Simulink® models. We evaluate ATheNA-S by considering a large set of models from different domains. Our results show that ATheNA-S generates more failure-revealing test cases than existing baseline tools and that the difference between the runtime performance of ATheNA-S and the baseline tools is not statistically significant. We also assess whether ATheNA-S could generate failure-revealing test cases when applied to two representative case studies: one from the automotive domain and one from the medical domain. Our results show that ATheNA-S successfully revealed a requirement violation in our case studies.
articolo
2024
Formica, Federico; Fan, Tony; Menghi, Claudio
(2024). Search-Based Software Testing Driven by Automatically Generated and Manually Defined Fitness Functions [journal article - articolo]. In ACM TRANSACTIONS ON SOFTWARE ENGINEERING AND METHODOLOGY. Retrieved from https://hdl.handle.net/10446/262251
File allegato/i alla scheda:
File Dimensione del file Formato  
TOSEM_CR.pdf

Solo gestori di archivio

Versione: postprint - versione referata/accettata senza referaggio
Licenza: Licenza default Aisberg
Dimensione del file 1.39 MB
Formato Adobe PDF
1.39 MB Adobe PDF   Visualizza/Apri
Pubblicazioni consigliate

Aisberg ©2008 Servizi bibliotecari, Università degli studi di Bergamo | Terms of use/Condizioni di utilizzo

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/10446/262251
Citazioni
  • Scopus ND
  • ???jsp.display-item.citation.isi??? ND
social impact