Towards Automated Prediction of Relationships Among Scientific Datasets

Published In

Proceedings of the 27th International Conference on Scientific and Statistical Database Management (SSDBM '15)

Document Type

Citation

Publication Date

2015

Subjects

Database management, Data mining, Information retrieval

Abstract

Before scientists can analyze, publish, or share their data, they often need to determine how their datasets are related. Determining relationships helps scientists identify the most complete version of a dataset, detect versions of datasets that complement each other, and determine multiple datasets that overlap. In previous work, we showed how observable relationships between two datasets help scientists recall their original derivation connection. While that work helped with identifying relationships between two datasets, it is infeasible for scientists to use it for finding relationships between all possible pairs in a large collection of datasets. In order to deal with larger numbers of datasets, we are extending our methodology with a relationship-prediction system, ReDiscover, a tool to identify pairs from a collection of datasets that are most likely related and the relationship between them. We report on the initial design of ReDiscover, which uses machine-learning methods such as Conditional Random Fields and Support Vector Machines to the relationship-discovery problem. Our preliminarily evaluation shows that ReDiscover predicted relationships with an average accuracy of 87%.

DOI

10.1145/2791347.2791358

Persistent Identifier

http://archives.pdx.edu/ds/psu/20877

Share

COinS