Mining academic publications to automatically identify data sources

Main Article Content

Athanasios Anastasiou
Karen Tingay


Discovering suitable datasets is an important part of health research, particularly for projects working with cohort data, but with the proliferation of so many national and international initiatives, it is becoming increasingly difficult for research teams to locate real world datasets that are most relevant to their project objectives.

To assist researchers in this, we developed bibInsight, a data analysis platform to identify potentially useful data sources and more generally enable large scale research over bibliographical datasets.

Data source names were identified from a broad, topic specific literature search. Context-specific terms like “annual”, “longitudinal”, and “prospective” were used to train a classifier that identified potential datasets.

The classifier was able to identify 1588 of 1961 abstracts as containing cohort-relevant information: a precision of approximately 80%.

Further analysis such as topic analysis, geographical mapping, and collaboration networks can refine and prioritise the search results to determine the most relevant data source(s) for a research project.

A very large amount of information, including data source description and use, remains unexploited in unstructured bibliographical datasets. Here, we used a thematic search to provide a more manageable starting point towards locating disease specific datasets.

Article Details

How to Cite
Anastasiou, A. and Tingay, K. (2018) “Mining academic publications to automatically identify data sources”, International Journal of Population Data Science, 3(2). doi: 10.23889/ijpds.v3i2.532.

Most read articles by the same author(s)

1 2 > >>