The University of Southampton
University of Southampton Institutional Repository

A survey on test suite reduction frameworks and tools

A survey on test suite reduction frameworks and tools
A survey on test suite reduction frameworks and tools
Software testing is a widely accepted practice that ensures the quality of a System under Test (SUT). However, the gradual increase of the test suite size demands high portion of testing budget and time. Test Suite Reduction (TSR) is considered a potential approach to deal with the test suite size problem. Moreover, a complete automation support is highly recommended for software testing to adequately meet the challenges of a resource constrained testing environment. Several TSR frameworks and tools have been proposed to efficiently address the test-suite size problem. The main objective of the paper is to comprehensively review the state-of-the-art TSR frameworks to highlights their strengths and weaknesses. Furthermore, the paper focuses on devising a detailed thematic taxonomy to classify existing literature that helps in understanding the underlying issues and proof of concept. Moreover, the paper investigates critical aspects and related features of TSR frameworks and tools based on a set of defined parameters. We also rigorously elaborated various testing domains and approaches followed by the extant TSR frameworks. The results reveal that majority of TSR frameworks focused on randomized unit testing, and a considerable number of frameworks lacks in supporting multi-objective optimization problems. Moreover, there is no generalized framework, effective for testing applications developed in any programming domain. Conversely, Integer Linear Programming (ILP) based TSR frameworks provide an optimal solution for multi-objective optimization problems and improve execution time by running multiple ILP in parallel. The study concludes with new insights and provides an unbiased view of the state-of-the-art TSR frameworks. Finally, we present potential research issues for further investigation to anticipate efficient TSR frameworks.
regression testing, test suite optimization, test suite reduction, frameworks, fault localization
0268-4012
963-975
Khan, Saif Ur Rehman
056226c9-3db8-4519-bc8c-3f285c8424f1
Lee, Sai Peck
787690ab-ecf8-45f5-afc3-873db4697d8a
Ahmad, Raja Wasim
347d92aa-40d9-4c90-a9e0-7a721779cd68
Akhunzada, Adnan
66b90cf7-8856-496e-9f08-e4bae7b596e9
Chang, Victor
a7c75287-b649-4a63-a26c-6af6f26525a4
Khan, Saif Ur Rehman
056226c9-3db8-4519-bc8c-3f285c8424f1
Lee, Sai Peck
787690ab-ecf8-45f5-afc3-873db4697d8a
Ahmad, Raja Wasim
347d92aa-40d9-4c90-a9e0-7a721779cd68
Akhunzada, Adnan
66b90cf7-8856-496e-9f08-e4bae7b596e9
Chang, Victor
a7c75287-b649-4a63-a26c-6af6f26525a4

Khan, Saif Ur Rehman, Lee, Sai Peck, Ahmad, Raja Wasim, Akhunzada, Adnan and Chang, Victor (2016) A survey on test suite reduction frameworks and tools. International Journal of Information Management, 36 (6), 963-975. (doi:10.1016/j.ijinfomgt.2016.05.025).

Record type: Article

Abstract

Software testing is a widely accepted practice that ensures the quality of a System under Test (SUT). However, the gradual increase of the test suite size demands high portion of testing budget and time. Test Suite Reduction (TSR) is considered a potential approach to deal with the test suite size problem. Moreover, a complete automation support is highly recommended for software testing to adequately meet the challenges of a resource constrained testing environment. Several TSR frameworks and tools have been proposed to efficiently address the test-suite size problem. The main objective of the paper is to comprehensively review the state-of-the-art TSR frameworks to highlights their strengths and weaknesses. Furthermore, the paper focuses on devising a detailed thematic taxonomy to classify existing literature that helps in understanding the underlying issues and proof of concept. Moreover, the paper investigates critical aspects and related features of TSR frameworks and tools based on a set of defined parameters. We also rigorously elaborated various testing domains and approaches followed by the extant TSR frameworks. The results reveal that majority of TSR frameworks focused on randomized unit testing, and a considerable number of frameworks lacks in supporting multi-objective optimization problems. Moreover, there is no generalized framework, effective for testing applications developed in any programming domain. Conversely, Integer Linear Programming (ILP) based TSR frameworks provide an optimal solution for multi-objective optimization problems and improve execution time by running multiple ILP in parallel. The study concludes with new insights and provides an unbiased view of the state-of-the-art TSR frameworks. Finally, we present potential research issues for further investigation to anticipate efficient TSR frameworks.

Text
IJIM_testt_suite_reduction_framework_accepted.pdf - Accepted Manuscript
Download (711kB)

More information

Accepted/In Press date: 28 May 2016
Published date: 1 December 2016
Keywords: regression testing, test suite optimization, test suite reduction, frameworks, fault localization
Organisations: Electronics & Computer Science, Electronic & Software Systems

Identifiers

Local EPrints ID: 396265
URI: http://eprints.soton.ac.uk/id/eprint/396265
ISSN: 0268-4012
PURE UUID: 350a372f-c24c-4c52-9a39-9535f0ca9e74

Catalogue record

Date deposited: 02 Jun 2016 02:59
Last modified: 07 Oct 2020 05:45

Export record

Altmetrics

Contributors

Author: Saif Ur Rehman Khan
Author: Sai Peck Lee
Author: Raja Wasim Ahmad
Author: Adnan Akhunzada
Author: Victor Chang

University divisions

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×