Evaluating Computer Science: Challenges and Solutions

Computer science, with conference articles as the dominant publication type, is a well-known challenge for university administrators and support services from the point of view of evaluation and bibliometric analyses. In the course of our work at Helsinki University Library, we have considered the available bibliometric databases, and also tried them in practice: Web of Science, Scopus, Google Scholar, Microsoft Academic 2.0 and Dimensions.

There is no perfect fit for computer science, as every database has its own problems. We provide a comparison, listing benefits and pitfalls. We also explain why we usually end up with choosing Google Scholar.

For evaluations including statistics from the Finnish Publication Forum classification, the changes announced for conference classification in June 2016 added a further dimension of complexity. For evaluations including publication years before and after this change, the results cannot be described as very transparent.

Can computer science be evaluated by following the principles of the Leiden Manifesto? We think that it is possible, provided that the evaluators let computer scientists have their say in the process. One should, however, be aware, that this process is not necessarily straightforward.