Towards a scalable, open standards service for cross-protocol data transfers across multiple sources an sinks


Meredith, David, Crouch, Stephen, Galang, Gerson, Jiang, Ming, Hung, Nguyen and Turner, Peter (2010) Towards a scalable, open standards service for cross-protocol data transfers across multiple sources an sinks. [in special issue: e-Science: past, present and future II] Philosophical Transactions of The Royal Society: A Mathematical Physical and Engineering Sciences, 368, (1926) (doi:10.1098/rsta.2010.0148).

Download

Full text not available from this repository.

Description/Abstract

Data Transfer Service (DTS) is an open-source project that is developing a document-centric message model for describing a bulk data transfer activity, with an accompanying set of loosely coupled and platform-independent components for brokering the transfer of data between a wide range of (potentially incompatible) storage resources as scheduled, fault-tolerant batch jobs. The architecture scales from small embedded deployments on a single computer to large distributed deployments through an expandable ‘worker-node pool’ controlled through message-orientated middleware. Data access and transfer efficiency are maximized through the strategic placement of worker nodes at or between particular data sources/sinks. The design is inherently asynchronous, and, when third-party transfer is not available, it side-steps the bandwidth, concurrency and scalability limitations associated with buffering bytes directly through intermediary client applications. It aims to address geographical–topological deployment concerns by allowing service hosting to be either centralized (as part of a shared service) or confined to a single institution or domain. Established design patterns and open-source components are coupled with a proposal for a document-centric and open-standards-based messaging protocol. As part of the development of the message protocol, a bulk data copy activity document is proposed for the first time.

Item Type: Article
ISSNs: 1364-503X (print)
1471-2962 (electronic)
Related URLs:
Subjects: B Philosophy. Psychology. Religion > B Philosophy (General)
Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Divisions: Faculty of Engineering and the Environment > Civil, Maritime and Environmental Engineering and Science
ePrint ID: 339157
Date Deposited: 24 May 2012 13:27
Last Modified: 28 Mar 2014 15:09
Research Funder: EPSRC
Projects:
OMII-UK Centre and Managed Programme
Funded by: EPSRC (EP/D076617/1)
Led by: David De Roure
1 July 2006 to 31 December 2010
URI: http://eprints.soton.ac.uk/id/eprint/339157

Actions (login required)

View Item View Item