The University of Southampton
University of Southampton Institutional Repository

Towards a scalable, open standards service for cross-protocol data transfers across multiple sources an sinks

Meredith, David, Crouch, Stephen, Galang, Gerson, Jiang, Ming, Hung, Nguyen and Turner, Peter (2010) Towards a scalable, open standards service for cross-protocol data transfers across multiple sources an sinks [in special issue: e-Science: past, present and future II] Philosophical Transactions of The Royal Society A, 368, (1926) (doi:10.1098/rsta.2010.0148).

Record type: Article

Abstract

Data Transfer Service (DTS) is an open-source project that is developing a document-centric message model for describing a bulk data transfer activity, with an accompanying set of loosely coupled and platform-independent components for brokering the transfer of data between a wide range of (potentially incompatible) storage resources as scheduled, fault-tolerant batch jobs. The architecture scales from small embedded deployments on a single computer to large distributed deployments through an expandable ‘worker-node pool’ controlled through message-orientated middleware. Data access and transfer efficiency are maximized through the strategic placement of worker nodes at or between particular data sources/sinks. The design is inherently asynchronous, and, when third-party transfer is not available, it side-steps the bandwidth, concurrency and scalability limitations associated with buffering bytes directly through intermediary client applications. It aims to address geographical–topological deployment concerns by allowing service hosting to be either centralized (as part of a shared service) or confined to a single institution or domain. Established design patterns and open-source components are coupled with a proposal for a document-centric and open-standards-based messaging protocol. As part of the development of the message protocol, a bulk data copy activity document is proposed for the first time.

Full text not available from this repository.

More information

Published date: 10 September 2010
Organisations: Civil Maritime & Env. Eng & Sci Unit

Identifiers

Local EPrints ID: 339157
URI: http://eprints.soton.ac.uk/id/eprint/339157
ISSN: 1364-503X
PURE UUID: cb9d778a-3382-4904-ad92-6cc17d6284db
ORCID for Stephen Crouch: ORCID iD orcid.org/0000-0001-8985-6814

Catalogue record

Date deposited: 24 May 2012 13:27
Last modified: 15 Sep 2017 16:34

Export record

Altmetrics

Contributors

Author: David Meredith
Author: Stephen Crouch ORCID iD
Author: Gerson Galang
Author: Ming Jiang
Author: Nguyen Hung
Author: Peter Turner

University divisions

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×