An approach to quantify integration quality using feedback on mapping results

Fernando R.S. Serrano, Alvaro A.A. Fernandes, Klitos Christodoulou

Research output: Contribution to journalArticle

Abstract

Purpose: The pay-as-you-go approach to data integration aims to reduce the time and effort required by proposing a bootstrap phase in which algorithms, rather than experts, identify semantic correspondences and generate the mappings. This highly automated bootstrap phase is likely to be of low quality, thus pay-as-you-go approaches postulate a subsequent continuous improvement phase based on user feedback assimilation to improve the quality of the integration. The purpose of this paper is to quantify the quality of a speculative integration, using one particular type of feedback, mapping results, whilst taking into account the uncertainty of user feedback provided. Design/methodology/approach: The authors propose a systematic approach to quantify the quality of an integration as a conditional probability given the trustworthiness of the workers. Given a set of mappings and a set of workers of unknown trustworthiness, feedback instances are collected in the extents of the mappings that characterize the integration. Taking into account the available evidence obtained from worker feedback, the technique provides a quality quantification of the speculative integration. Findings: Experimental results on both synthetic and real-world scenarios provide valuable empirical evidence that the technique produces a cost-effective quantification of integration quality that faithfully reflects the judgement of the workers whilst taking into account the inherent uncertainty of user feedback. Originality/value: Current pay-as-you-go techniques provide a limited view of the integration quality as the result of feedback assimilation. To the best of the authors’ knowledge, this is the first proposal for quantifying integration quality in a systematic and principled manner using mapping results as a piece of evidence while at the same time considering the uncertainty inherited from user feedback.

Original languageEnglish
Pages (from-to)47-70
Number of pages24
JournalInternational Journal of Web Information Systems
Volume15
Issue number1
DOIs
Publication statusPublished - 4 Apr 2019
Externally publishedYes

Fingerprint

Feedback
Data integration
Semantics
Uncertainty
Costs

Keywords

  • Data integration
  • Feedback assimilation
  • Pay-as-you-go

Cite this

@article{8341f9c0a62d4a009137251fe0a059af,
title = "An approach to quantify integration quality using feedback on mapping results",
abstract = "Purpose: The pay-as-you-go approach to data integration aims to reduce the time and effort required by proposing a bootstrap phase in which algorithms, rather than experts, identify semantic correspondences and generate the mappings. This highly automated bootstrap phase is likely to be of low quality, thus pay-as-you-go approaches postulate a subsequent continuous improvement phase based on user feedback assimilation to improve the quality of the integration. The purpose of this paper is to quantify the quality of a speculative integration, using one particular type of feedback, mapping results, whilst taking into account the uncertainty of user feedback provided. Design/methodology/approach: The authors propose a systematic approach to quantify the quality of an integration as a conditional probability given the trustworthiness of the workers. Given a set of mappings and a set of workers of unknown trustworthiness, feedback instances are collected in the extents of the mappings that characterize the integration. Taking into account the available evidence obtained from worker feedback, the technique provides a quality quantification of the speculative integration. Findings: Experimental results on both synthetic and real-world scenarios provide valuable empirical evidence that the technique produces a cost-effective quantification of integration quality that faithfully reflects the judgement of the workers whilst taking into account the inherent uncertainty of user feedback. Originality/value: Current pay-as-you-go techniques provide a limited view of the integration quality as the result of feedback assimilation. To the best of the authors’ knowledge, this is the first proposal for quantifying integration quality in a systematic and principled manner using mapping results as a piece of evidence while at the same time considering the uncertainty inherited from user feedback.",
keywords = "Data integration, Feedback assimilation, Pay-as-you-go",
author = "Serrano, {Fernando R.S.} and Fernandes, {Alvaro A.A.} and Klitos Christodoulou",
year = "2019",
month = "4",
day = "4",
doi = "10.1108/IJWIS-05-2018-0043",
language = "English",
volume = "15",
pages = "47--70",
journal = "International Journal of Cooperative Information Systems",
issn = "0218-8430",
publisher = "World Scientific Publishing Co. Pte Ltd",
number = "1",

}

An approach to quantify integration quality using feedback on mapping results. / Serrano, Fernando R.S.; Fernandes, Alvaro A.A.; Christodoulou, Klitos.

In: International Journal of Web Information Systems, Vol. 15, No. 1, 04.04.2019, p. 47-70.

Research output: Contribution to journalArticle

TY - JOUR

T1 - An approach to quantify integration quality using feedback on mapping results

AU - Serrano, Fernando R.S.

AU - Fernandes, Alvaro A.A.

AU - Christodoulou, Klitos

PY - 2019/4/4

Y1 - 2019/4/4

N2 - Purpose: The pay-as-you-go approach to data integration aims to reduce the time and effort required by proposing a bootstrap phase in which algorithms, rather than experts, identify semantic correspondences and generate the mappings. This highly automated bootstrap phase is likely to be of low quality, thus pay-as-you-go approaches postulate a subsequent continuous improvement phase based on user feedback assimilation to improve the quality of the integration. The purpose of this paper is to quantify the quality of a speculative integration, using one particular type of feedback, mapping results, whilst taking into account the uncertainty of user feedback provided. Design/methodology/approach: The authors propose a systematic approach to quantify the quality of an integration as a conditional probability given the trustworthiness of the workers. Given a set of mappings and a set of workers of unknown trustworthiness, feedback instances are collected in the extents of the mappings that characterize the integration. Taking into account the available evidence obtained from worker feedback, the technique provides a quality quantification of the speculative integration. Findings: Experimental results on both synthetic and real-world scenarios provide valuable empirical evidence that the technique produces a cost-effective quantification of integration quality that faithfully reflects the judgement of the workers whilst taking into account the inherent uncertainty of user feedback. Originality/value: Current pay-as-you-go techniques provide a limited view of the integration quality as the result of feedback assimilation. To the best of the authors’ knowledge, this is the first proposal for quantifying integration quality in a systematic and principled manner using mapping results as a piece of evidence while at the same time considering the uncertainty inherited from user feedback.

AB - Purpose: The pay-as-you-go approach to data integration aims to reduce the time and effort required by proposing a bootstrap phase in which algorithms, rather than experts, identify semantic correspondences and generate the mappings. This highly automated bootstrap phase is likely to be of low quality, thus pay-as-you-go approaches postulate a subsequent continuous improvement phase based on user feedback assimilation to improve the quality of the integration. The purpose of this paper is to quantify the quality of a speculative integration, using one particular type of feedback, mapping results, whilst taking into account the uncertainty of user feedback provided. Design/methodology/approach: The authors propose a systematic approach to quantify the quality of an integration as a conditional probability given the trustworthiness of the workers. Given a set of mappings and a set of workers of unknown trustworthiness, feedback instances are collected in the extents of the mappings that characterize the integration. Taking into account the available evidence obtained from worker feedback, the technique provides a quality quantification of the speculative integration. Findings: Experimental results on both synthetic and real-world scenarios provide valuable empirical evidence that the technique produces a cost-effective quantification of integration quality that faithfully reflects the judgement of the workers whilst taking into account the inherent uncertainty of user feedback. Originality/value: Current pay-as-you-go techniques provide a limited view of the integration quality as the result of feedback assimilation. To the best of the authors’ knowledge, this is the first proposal for quantifying integration quality in a systematic and principled manner using mapping results as a piece of evidence while at the same time considering the uncertainty inherited from user feedback.

KW - Data integration

KW - Feedback assimilation

KW - Pay-as-you-go

UR - http://www.scopus.com/inward/record.url?scp=85059324147&partnerID=8YFLogxK

U2 - 10.1108/IJWIS-05-2018-0043

DO - 10.1108/IJWIS-05-2018-0043

M3 - Article

VL - 15

SP - 47

EP - 70

JO - International Journal of Cooperative Information Systems

JF - International Journal of Cooperative Information Systems

SN - 0218-8430

IS - 1

ER -