Unsupervised semantic similarity computation between terms using web documents

Elias Iosif, Alexandros Potamianos

Research output: Contribution to journalArticlepeer-review

Abstract

In this work, Web-based metrics that compute the semantic similarity between words or terms are presented and compared with the state of the art. Starting from the fundamental assumption that similarity of context implies similarity of meaning, relevant Web documents are downloaded via a Web search engine and the contextual information of words of interest is compared (context-based similarity metrics). The proposed algorithms work automatically, do not require any human-annotated knowledge resources, e.g., ontologies, and can be generalized and applied to different languages. Context-based metrics are evaluated both on the Charles-Miller data set and on a medical term data set. It is shown that context-based similarity metrics significantly outperform co-occurrence-based metrics, in terms of correlation with human judgment, for both tasks. In addition, the proposed unsupervised context-based similarity computation algorithms are shown to be competitive with the state-of-the-art supervised semantic similarity algorithms that employ language-specific knowledge resources. Specifically, context-based metrics achieve correlation scores of up to 0.88 and 0.74 for the Charles-Miller and medical data sets, respectively. The effect of stop word filtering is also investigated for word and term similarity computation. Finally, the performance of context-based term similarity metrics is evaluated as a function of the number of Web documents used and for various feature weighting schemes.

Original languageEnglish
Article number5291692
Pages (from-to)1637-1647
Number of pages11
JournalIEEE Transactions on Knowledge and Data Engineering
Volume22
Issue number11
DOIs
Publication statusPublished - 2010
Externally publishedYes

Keywords

  • knowledge acquisition
  • Natural language processing
  • ontologies
  • semantic similarity
  • Web search

Fingerprint

Dive into the research topics of 'Unsupervised semantic similarity computation between terms using web documents'. Together they form a unique fingerprint.

Cite this