Adapting integrity enforcement techniques for data reconciliation

Download

PDF Restricted to Registered users only Download (883Kb) | Request a copy

Description/Abstract

Integration of data sources opens up possibilities for new and valuable applications of data that cannot be supported
by the individual sources alone. Unfortunately, many data integration projects are hindered by the inherent
heterogeneities in the sources to be integrated. In particular, differences in the way that real world data is encoded
within sources can cause a range of difficulties, not least of which is that the conflicting semantics may not be recognised
until the integration project is well under way. Once identified, semantic conflicts of this kind are typically dealt with by
configuring a data transformation engine, that can convert incoming data into the form required by the integrated
system. However, determination of a complete and consistent set of data transformations for any given integration task
is far from trivial. In this paper, we explore the potential application of techniques for integrity enforcement in
supporting this process. We describe the design of a data reconciliation tool (LITCHI) based on these techniques that
aims to assist taxonomists in the integration of biodiversity data sets. Our experiences have highlighted several
limitations of integrity enforcement when applied to this real world problem, and we describe how we have overcome
these in the design of our system.