Towards a scalable, open standards service for cross-protocol data transfers across multiple sources an sinks
Meredith, David, Crouch, Stephen, Galang, Gerson, Jiang, Ming, Hung, Nguyen and Turner, Peter (2010) Towards a scalable, open standards service for cross-protocol data transfers across multiple sources an sinks. [in special issue: e-Science: past, present and future II] Philosophical Transactions of The Royal Society: A Mathematical Physical and Engineering Sciences, 368, (1926) (doi:10.1098/rsta.2010.0148).
Download
Full text not available from this repository.
Description/Abstract
Data Transfer Service (DTS) is an open-source project that is developing a document-centric message model for describing a bulk data transfer activity, with an accompanying set of loosely coupled and platform-independent components for brokering the transfer of data between a wide range of (potentially incompatible) storage resources as scheduled, fault-tolerant batch jobs. The architecture scales from small embedded deployments on a single computer to large distributed deployments through an expandable ‘worker-node pool’ controlled through message-orientated middleware. Data access and transfer efficiency are maximized through the strategic placement of worker nodes at or between particular data sources/sinks. The design is inherently asynchronous, and, when third-party transfer is not available, it side-steps the bandwidth, concurrency and scalability limitations associated with buffering bytes directly through intermediary client applications. It aims to address geographical–topological deployment concerns by allowing service hosting to be either centralized (as part of a shared service) or confined to a single institution or domain. Established design patterns and open-source components are coupled with a proposal for a document-centric and open-standards-based messaging protocol. As part of the development of the message protocol, a bulk data copy activity document is proposed for the first time.
| Item Type: | Article |
|---|---|
| ISSNs: | 1364-503X (print) 1471-2962 (electronic) |
| Related URLs: | |
| Subjects: | B Philosophy. Psychology. Religion > B Philosophy (General) Q Science > QA Mathematics > QA75 Electronic computers. Computer science |
| Divisions: | Faculty of Engineering and the Environment > Civil, Maritime and Environmental Engineering and Science |
| Item ID: | 339157 |
| Date Deposited: | 24 May 2012 13:27 |
| Last Modified: | 26 Apr 2013 06:01 |
| Contributors: | Meredith, David (Author) Crouch, Stephen (Author) Galang, Gerson (Author) Jiang, Ming (Author) Hung, Nguyen (Author) Turner, Peter (Author) |
| Date: | 10 September 2010 |
| Status: | Published |
| URI: | http://eprints.soton.ac.uk/id/eprint/339157 |
Actions (login required)
 |
View Item |
