The University of Southampton
University of Southampton Institutional Repository

An On-the-fly Provenance Tracking Mechanism for Stream Processing Systems

An On-the-fly Provenance Tracking Mechanism for Stream Processing Systems
An On-the-fly Provenance Tracking Mechanism for Stream Processing Systems
Applications that operate over streaming data with
high-volume and real-time processing requirements are becoming
increasingly important. These applications process streaming
data in real-time and deliver instantaneous responses to support
precise and on-time decisions. In such systems, traceability -
the ability to verify and investigate the source of a particular
output - in real-time is extremely important. This ability allows
raw streaming data to be checked and processing steps to be
verified and validated in timely manner. Therefore, it is crucial
that stream systems have a mechanism for dynamically tracking
provenance - the process that produced result data - at execution
time, which we refer to as on-the-fly stream provenance tracking.
In this paper, we propose a novel on-the-fly provenance tracking
mechanism that enables provenance queries to be performed
dynamically without requiring provenance assertions to be stored
persistently. We demonstrate how our provenance mechanism
works by means of an on-the-fly provenance tracking algorithm.
The experimental evaluation shows that our provenance solution
does not have a significant effect on the normal processing of
stream systems given a 7% overhead. Moreover, our provenance
solution offers low-latency processing (0.3 ms per additional
component) with reasonable memory consumption.
Sansrimahachai, Watsawee
49e185c7-e55c-4c40-9d97-ca7b861d7e94
Weal, Mark J.
e8fd30a6-c060-41c5-b388-ca52c81032a4
Moreau, Luc
033c63dd-3fe9-4040-849f-dfccbe0406f8
Sansrimahachai, Watsawee
49e185c7-e55c-4c40-9d97-ca7b861d7e94
Weal, Mark J.
e8fd30a6-c060-41c5-b388-ca52c81032a4
Moreau, Luc
033c63dd-3fe9-4040-849f-dfccbe0406f8

Sansrimahachai, Watsawee, Weal, Mark J. and Moreau, Luc (2013) An On-the-fly Provenance Tracking Mechanism for Stream Processing Systems. IEEE/ACIS International Conference on Computer and Information Science (ICIS 2013). (doi:10.1109/ICIS.2013.6607885).

Record type: Article

Abstract

Applications that operate over streaming data with
high-volume and real-time processing requirements are becoming
increasingly important. These applications process streaming
data in real-time and deliver instantaneous responses to support
precise and on-time decisions. In such systems, traceability -
the ability to verify and investigate the source of a particular
output - in real-time is extremely important. This ability allows
raw streaming data to be checked and processing steps to be
verified and validated in timely manner. Therefore, it is crucial
that stream systems have a mechanism for dynamically tracking
provenance - the process that produced result data - at execution
time, which we refer to as on-the-fly stream provenance tracking.
In this paper, we propose a novel on-the-fly provenance tracking
mechanism that enables provenance queries to be performed
dynamically without requiring provenance assertions to be stored
persistently. We demonstrate how our provenance mechanism
works by means of an on-the-fly provenance tracking algorithm.
The experimental evaluation shows that our provenance solution
does not have a significant effect on the normal processing of
stream systems given a 7% overhead. Moreover, our provenance
solution offers low-latency processing (0.3 ms per additional
component) with reasonable memory consumption.

Text
OnTheFlyProvenance_CameraReady.pdf - Accepted Manuscript
Download (0B)

More information

Published date: June 2013
Organisations: Agents, Interactions & Complexity

Identifiers

Local EPrints ID: 357066
URI: http://eprints.soton.ac.uk/id/eprint/357066
PURE UUID: 3fff0bf5-d16f-4356-bec6-49a4303ff5cf
ORCID for Mark J. Weal: ORCID iD orcid.org/0000-0001-6251-8786
ORCID for Luc Moreau: ORCID iD orcid.org/0000-0002-3494-120X

Catalogue record

Date deposited: 18 Sep 2013 21:25
Last modified: 29 Oct 2019 02:07

Export record

Altmetrics

Contributors

Author: Watsawee Sansrimahachai
Author: Mark J. Weal ORCID iD
Author: Luc Moreau ORCID iD

University divisions

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×