Towards Automated Prediction of Relationships Among Scientific Datasets

Published In

Proceedings of the 27th International Conference on Scientific and Statistical Database Management (SSDBM '15)

Document Type


Publication Date



Database management, Data mining, Information retrieval


Before scientists can analyze, publish, or share their data, they often need to determine how their datasets are related. Determining relationships helps scientists identify the most complete version of a dataset, detect versions of datasets that complement each other, and determine multiple datasets that overlap. In previous work, we showed how observable relationships between two datasets help scientists recall their original derivation connection. While that work helped with identifying relationships between two datasets, it is infeasible for scientists to use it for finding relationships between all possible pairs in a large collection of datasets. In order to deal with larger numbers of datasets, we are extending our methodology with a relationship-prediction system, ReDiscover, a tool to identify pairs from a collection of datasets that are most likely related and the relationship between them. We report on the initial design of ReDiscover, which uses machine-learning methods such as Conditional Random Fields and Support Vector Machines to the relationship-discovery problem. Our preliminarily evaluation shows that ReDiscover predicted relationships with an average accuracy of 87%.

Locate the Document

PSU affiliates use Find in PSU library link at top.

Unaffiliated researchers can access the work here: http://dx.doi.org/10.1145/2791347.2791358



Persistent Identifier