The University of Southampton
University of Southampton Institutional Repository

Fast, fully-automated, model-based fault localisation and repair with test suites as specification

Fast, fully-automated, model-based fault localisation and repair with test suites as specification
Fast, fully-automated, model-based fault localisation and repair with test suites as specification
Fault localisation, i.e. the identification of program locations that cause errors, takes significant effort and cost. We describe a fast model-based fault localisation algorithm which, given a test suite, uses symbolic execution methods to fully automatically identify a small subset of program locations where genuine program repairs exist. Our algorithm iterates over failing test cases and collects locations where an assignment change can repair exhibited faulty behaviour. Our main contribution is an improved search through the test suite, reducing the effort for the symbolic execution of the models and leading to speed-ups of more than two orders of magnitude over previously published work.

We implemented our algorithm for C programs, using the KLEE symbolic execution engine, and demonstrate its effectiveness on the Siemens TCAS variants. Its performance is in line with recent alternative model-based fault localisation techniques, but narrows the location set further without rejecting any genuine repair locations. We explore extending the low-quality repairs constructed by the localisation process to synthesise a high-quality repair.

We also show how our tool can be used in an educational context to improve selfguided learning and accelerate assessment. We apply our algorithm to a large selection of actual student coursework submissions, providing precise localisation within a subsecond response time. We show this using small test suites, already provided in the coursework management system, and on expanded test suites, demonstrating scaling. We also show that compliance with test suites does not predictably score a class of "almost correct" submissions, which our tool highlights. Finally, we show an extension to our tool which enables a selection of student submissions to be localised beyond programs that conform to a single-fault assumption.
University of Southampton
Birch, Geoff Michael
4e118f9f-4a3a-4f28-893d-57423b360c16
Birch, Geoff Michael
4e118f9f-4a3a-4f28-893d-57423b360c16
Poppleton, Michael
4c60e63f-188c-4636-98b9-de8a42789b1b

Birch, Geoff Michael (2016) Fast, fully-automated, model-based fault localisation and repair with test suites as specification. University of Southampton, Physical Sciences and Engineering, Doctoral Thesis, 123pp.

Record type: Thesis (Doctoral)

Abstract

Fault localisation, i.e. the identification of program locations that cause errors, takes significant effort and cost. We describe a fast model-based fault localisation algorithm which, given a test suite, uses symbolic execution methods to fully automatically identify a small subset of program locations where genuine program repairs exist. Our algorithm iterates over failing test cases and collects locations where an assignment change can repair exhibited faulty behaviour. Our main contribution is an improved search through the test suite, reducing the effort for the symbolic execution of the models and leading to speed-ups of more than two orders of magnitude over previously published work.

We implemented our algorithm for C programs, using the KLEE symbolic execution engine, and demonstrate its effectiveness on the Siemens TCAS variants. Its performance is in line with recent alternative model-based fault localisation techniques, but narrows the location set further without rejecting any genuine repair locations. We explore extending the low-quality repairs constructed by the localisation process to synthesise a high-quality repair.

We also show how our tool can be used in an educational context to improve selfguided learning and accelerate assessment. We apply our algorithm to a large selection of actual student coursework submissions, providing precise localisation within a subsecond response time. We show this using small test suites, already provided in the coursework management system, and on expanded test suites, demonstrating scaling. We also show that compliance with test suites does not predictably score a class of "almost correct" submissions, which our tool highlights. Finally, we show an extension to our tool which enables a selection of student submissions to be localised beyond programs that conform to a single-fault assumption.

Text
thesis (final).pdf - Version of Record
Available under License University of Southampton Thesis Licence.
Download (1MB)

More information

Published date: December 2016

Identifiers

Local EPrints ID: 405470
URI: http://eprints.soton.ac.uk/id/eprint/405470
PURE UUID: a5adf3bc-bae0-4e7e-be69-31bc84a5dd27

Catalogue record

Date deposited: 18 Feb 2017 00:22
Last modified: 11 Nov 2020 17:32

Export record

Contributors

Author: Geoff Michael Birch
Thesis advisor: Michael Poppleton

University divisions

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×