Cluster Management of Scientific Literature in HSTOOL


  • Johan Schubert
  • Ulrika Wickenberg Bolin

Publish date: 2021-11-03

Report number: FOI-R--5178--SE

Pages: 18

Written in: English

Research areas:

  • Övrigt


  • horizon scanning
  • scientometrics
  • Gibbs sampling
  • Dirichlet multinomial mixture model
  • entropy
  • clustering


In this report, we expand a methodology for horizon scanning of scientific literature to discover scientific trends. In this methodology, scientific articles are automatically clustered within a broadly defined field of research based on the topic. We develop a new method to allow an analyst to handle the large number of clusters that result from the automatic clustering of articles. The method is based on estimating an informationtheoretical distance between all possible pairs of clusters. Each of the scientific articles has a probability distribution of affiliation over all possible clusters arising from the clustering process. Using these, we investigate possible pairwise mergers between all pairs of existing clusters and calculate the entropies of the probability distributions of all articles after each possible merger of two clusters. These entropies are visualized in a dendritic tree and a cluster graph. The merger with minimal total entropy is the proposed cluster pair to be merged.

Share page on social media