Making Use of Bibliometrics: What, and Why?

A citation score is a measure of attention for a publication, which does not necessarily reflect the quality of the research.

Citations are most easily measured for journal articles, although it is also possible to quantify the attention for other forms of research output such as books, theses and datasets.

Citation scores can be analysed for individual articles, an entire journal, or the complete works of an author or research organisation.

Citation data which is free to access:

• Google Scholar records incorporate a citation score generated automatically from Google's processes for indexing bibliographic records.
• The Dimensions database (owned by Digital Sciences) incorporates citation data derived from the connections between records in the database.  Dimensions data is free to access for personal non-commercial use.

Subscription-based sources of citation data:

• Web of Science (owned by Clarivate) and Scopus and SciVal (both owned by Elsevier) are the best known sources of curated citation data, for subscribing institutions.

Each database providing citation data indexes a slightly different set of publications, and the metrics are derived in different ways. It's not good practice to quote a citation score without identifying the source.  You should never rank publications, authors or journals using citation analytics which have been derived from different databases. 

For sources of bibliometric data which are not wholly based on citation counts, see Alternative Metrics.

Analysis of bibliometric data can help researchers:

• Demonstrate the scholarly impact of their own published work
• Identify the most prominent researchers in a field
• Decide which journals to read or submit papers to
• Develop a research strategy for a group or institution.

However, relying on citation scores without any element of peer review or contextual knowledge may lead to a very simplistic or incomplete understanding of the research.  The University of Hull Statement on the Responsible Use of Metrics recognises the limitations of bibliometric analysis (see below).

The University of Hull has published a Statement on the Use of Metrics in the Assessment of Research (2020).

Some key points:

• Decision-making must never be based entirely on quantitative measures, and should always support and not supplant expert peer assessment.
• The University’s use of metrics will be based upon the Leiden Manifesto (2015), and informed by The Metric Tide (Research England, 2014) and UK Progress Towards the Use of Metrics Responsibly (2018).
• Data being collected or acquired to provide the basis for metrics should be open and transparent so all those involved can understand what the data is and how and why it will be used.
• Metrics that provide excessive detail in their precision should also be avoided so metrics are not used simply for the sake of measuring.
• The metrics identified will benefit from regular scrutiny to ensure they continue to be fit for purpose over time, and changed if they are not.
• Faculties and Institutes are invited to use this document to inform the development of their own metrics toolset.
• A decision on whether to sign the San Francisco Declaration on Research Assessment (DORA) or not is delegated to those disciplines that can best utilise DORA as part of their metrics toolset.

• Support from the University Library for sourcing and interpreting bibliometric data:  rdm@hull.ac.uk
  • Kirstyn Radford, Research Outputs Specialist, and Chris Awre, Head of Information Services

• Guidance on the University's position on the use of bibliometrics for research evaluation is provided by the Research and Innovation Office.