NWB2023_On the performativity of SDG classifications in large bibliometric databases

Large bibliometric databases as digital infrastructures not only facilitate bibliometric analyses but are themselves performative. They are built upon a particular understanding of the science system and resulting attribution of worth, which affect the visibility of scientific outputs and impact measurement of participating entities like individual authors or institutions.

Initialised by science policy the contribution of the science system and its entities to the UN’s SDGs has recently gained much attention in the bibliometric impact debate. Web of Science, Scopus and Dimensions have all provided a respective SDG classification of their indexed publications to facilitate a respective measurement. Given the performative character of the diverse bibliometric databases it was quickly noted that these classifications do not match (Armitage et al., 2020). Still the underlying reasons for the observed differences and hence an insight on how bibliometric classifications of publications are themselves performative are still investigated.

At the same time the recently developed technology of large language models (LLM) has been criticised for their missing objectivity carrying forward data biases into the generated answers. In this work-in-progress we propose to utilise this particular feature of LLM to learn about the “data bias” injected by the diverse SDG classifications into the bibliometric data. Hence we present a LLM of jointly indexed publications, which we separately fine-tune by the diverse SDG classifications. A qualitative text analyses of the generated answers shows the inscribed understanding of the varying SDG classification and can be applied to inform science policy on the diverse SDG classifications.