Data-intensive approaches to finding and predicting research outcomes for New Zealand health research

Presentation delivered at the eResearch New Zealand 2020 conference in Dunedin, February 14, 2020.

--

How can we use data science to measure research outcomes at scale? Can quantitative data be used at all to understand research’s “impact” in its truest sense? In this presentation, we will share how—by asking the right questions, using the right data, and understanding data science’s strengths and limitations—New Zealanders can measure their success towards achieving health research outcomes, and even forecast future success.

Using the New Zealand Health Ministry’s “New Zealand Health Research Strategy 2017-2027” report as a case study, we will first show how thoughtful strategic planning makes it possible for data scientists to answer pressing questions like, “How can we track the implementation of research into health policy?” and “How can we produce the best research that supports the well-being of all New Zealanders?”

Next, we will discuss how linked bibliometric and altmetric data sources can help analysts better understand if and how New Zealand health research has achieved strategic priorities. Using unique data from Dimensions Analytics, a linked research intelligence database, and Altmetric Explorer, which provides data for understanding the broader impacts of research, we will use large scale visualization and statistical approaches to understand the current state of New Zealand health research with regard to desired outcomes; predict future trends based on past funding and publishing activity; and offer suggestions for ways to improve the likelihood of achieving desired research outcomes in the future. Among the outcomes studied will be international and industry collaboration trends, the translation of research into innovation and public policy, and public engagement with health research.

Finally, recapping the benefits and limitations of quantitative data in measuring desired outcomes like community collaborations and whether research is improving health outcomes for Maōri and disabled peoples. In some cases, leading engagement indicators like altmetrics can be used as a rough proxy for success, and are complementary to traditional program evaluation approaches. We explore several instances where altmetric and bibliometric data succeed and where they fail.

References

Ministry of Business, Innovation and Employment and Ministry of Health. 2017. New Zealand Health Research Strategy 2017-2027. Wellington: Ministry of Business, Innovation and Employment and Ministry of Health. Retrieved from: https://www.health.govt.nz/publication/new-zealand-health-research-strategy-2017-2027