Last week I was fortunate enough to attend two workshops that gave me the opportunity to reflect on open research and its contribution to research integrity. One was a stimulating afternoon at the BMJ to consider the future of publishing in health research, and the other was the first meeting of the UK Research Integrity Forum. At the latter, I spoke briefly about the contribution of open research to research integrity and, in particular, to reproducibility. Here’s the main slide from that talk, in case that’s useful.
One of my lines of thought since then has been about trust in research. I am not a veteran of the 1990s “science wars” (though I did dabble), that descended into a binary argument between social constructivists and realists on the epistemology and ontology of science. Those arguments seem both rather innocent and a little prescient in today’s world of social media, fake news and alternative facts. Rick Anderson has written about this with respect to libraries, though I disagree with his assumption that you need to be realist to counter lies. But here I want to think more broadly than libraries.
If:
- the world outside research is becoming increasingly relativist, the web increasingly centralised, and information increasingly siloed by the filter bubble; and
- research is suffering a reproducibility crisis…
…Then should we be worried about a perfect storm for trust in research? While trust in “experts” is reportedly threatened, at the moment (if you trust the figures) trust in scientists still high. However, I wonder how brittle that is? I wonder how many stories of irreproducible research it would take for trust in scientists to fall as low as trust is now in politicians or journalists.
One question I raised at the Research Integrity Forum was whether, in these circumstances, we need more and better public engagement with research, for example to work with society to arrive at better mutual understandings of risk and probability? And we may need to have a more informed conversation about how credit is given to researchers for the positive impact of research, and how we handle negative outcomes (job losses, environmental damage) where these are seen by some to be associated with science.
To bring those rather general questions a bit closer to my work at Jisc, these seem like important and urgent drivers for open research, which we’ve championed and supported for a long time. That is not a blanket call for all aspects of research to be open, but instead – and we believe in line with the national and international approaches:
- Sustainable open access to publications (OA)
- Intelligent open access to research data, as defined in the 2012 Royal Society report “Science as an Open Enterprise”, noting constraints such as sensitive data.
- More responsible use of more transparent research indicators
Open research is on the minds of others, too. The Royal Society’s motto is “Nullius in verba”, or take nobody’s word for it, and this seems in tune with the times, though not necessarily with everything that goes on in research, which may be one of the reasons why the Society is looking so hard at research culture at the moment. They are not the only ones imagining futures, though, and the BMJ’s scenarios for the future of global research also offer insight. More specifically on open research and reproducibility, there is a burgeoning literature on the latter, which perhaps shows some signs of being seized by the research community as an opportunity both for community ownership of the challenge, and for some really ground-breaking interdisciplinary meta-research. However, there are so many important questions that, as far as I can see, remain without convincing answers, for example:
- What does “reproducibility” look like in different disciplines? At present the heavy lifting has perhaps been done by psychology and some areas of the biosciences, but we need a much broader understanding. For example, there are some different views from physics here [PDF], economics here, and energy research here.
- Exactly how and when does open research contribute to research reproducibility and the integrity of the scholarly record in those disciplines? Of course there is the excellent work associated with the Center for Open Science (for example, see here), and some indications that openness and statistical reporting quality are related. However, given that an increasing amount of research is now open, we should have more and better evidence of its contribution to research reproducibility.
- And how does all the above work for interdisciplinary research, with its particular challenges around incommensurability of language, concepts and even epistemology? Such challenges are likely to be common in mission-oriented, challenge-based research such as that funded under the UK Industrial Strategy Challenge Fund, and advocated by Mazzucato for FP9 and supported – up to a point – by a recent UK Government position paper [PDF].
- As research moves increasingly online, can it avoid adopting some of the norms of that world? What are the risks to research integrity from the social media firestorm? We see increasing attention to altmetrics for research, but we know how easily those can be gamed in the wider web. We know how large online platforms monitor our every interaction and use these data as the basis for their business model. What is the basis for trust in this environment, and are there technological fixes such as blockchain for science?
Jisc does not have answers to these questions, but we do have a deep interest in knowing the answers, to fulfil our mission to UK research. That’s one reason for the national and international partnerships we have.
