This is a guest blog post from Dr Sam Westwood, Lecturer in Psychology at the University of Westminster. Sam founded and runs the RIOT Science Club (https://riotscience.co.uk/), a forum where researchers can learn about Open Research reforms and practices. The RIOT Science club aims to foster Reproducible, Interpretable, Open & Transparent Science. Sam also runs ReproducibiliTea at Westminster.
Catch up on the RIOTS Club seminar series on YouTube and follow the RIOT Science Club on twitter.
This blog was originally published in October 2022 on the British Neuroscience Association blog and was reposted with permission from the BNA. The blog was written on the occasion of a Festschrift for Dorothy Bishop, Professor of Developmental Neuropsychology at the University of Oxford.
The problem
The shift toward open science is thanks to both top-down and bottom-up initiatives, including the UK Reproducibility Network (UKRN), Open Science Framework (OSF), Declaration on Research Assessment (DORA), Reproducible, Interpretable, Open and Transparent (RIOT) Science Club, Framework for Open and Reproducible Research Training (FORRT) and ReproducibiliTea journal clubs – all championed by a groundswell of ECRs with support or leadership by example from a relative minority of senior academics, including Dorothy Bishop. Thanks to this collective action, open science is now becoming mainstream in many areas of research.
Like with anything else, however, far-reaching changes in research culture can introduce unintended consequences, some of which we are seeing now. One discussed during the Festschrift was that ECRs are invariably the advocates and practitioners of open science, yet decision-making in hiring and promotion panels, peer review, and grant funding bodies does not place enough value on open science practices, thereby disadvantaging junior adoptees of open science just starting out in their career. Traditional metrics, such as statistically significant findings, novelty, grant-funding capture, number of publications, journal impact factors, and h-index still unduly determine career progression. This problem – where open science practices such as pre-registration and data/code sharing are not appropriately and routinely rewarded – will undermine the sustainability of the commitment of future generations of academics to open science. The risk would be that open science and broader efforts to improve research integrity will only raise awareness of the common issues but may not lead to substantial real-world positive change in the longer-term.
The cost of not reforming traditional metrics not only undermines research quality and public trust in academic research, but it also exacerbates the brain drain in academia. University sector-wide cultural norms, such as prohibitive workloads and erosion of income by inflation or pension cuts, are forcing would-be academics to choose alternative career paths irrespective of career stage, whether to improve their living standard and/or their mental wellbeing. Add to this the fact that many skills under the open science umbrella (e.g., reproducible research, data wrangling, visualisation, analysis, big team science, secondary data analysis) are often handsomely valued outside of academia such as in data science, programming, and user experience research, then it will be no surprise if the bottom-heavy support that open science has enjoyed from ECRs begins to slow or dry up.
Possible solutions
The good news, however, is that there are plenty of ways to improve how open science is valued and thus to ensure that the talented ECRs engaged in open science have the opportunity to pursue an academic career path without making personal or professional sacrifices. After all, the ECRs of today are the academic leaders of tomorrow. For institutions, this can be done by increasingly recognising and funding grassroots initiatives that, so far, have mostly been run by ECRs on a volunteer basis. Some universities are leading the way by implementing updated open and reproducible policies and offering new job positions for consultancy and training of open and reproducible practices. Furthermore, institutions must take steps towards recognising the competitive advantages of candidates with experiences surrounding open science. It is imperative for universities and funding bodies to include it among criteria in decision-making processes by updating hiring and promotion panels and peer review. Some examples of such criteria may include knowledge and/or experience with protocol registration, open data management and/or reproducible workflows.
On the other hand, senior academics can help to sustain the progression of open and reproducible science by offering mentorship and support to ECRs. Professor Dorothy Bishop is an excellent role model in this regard. She leads by example, adopting and popularising solutions for improved integrity and transparency of research. She inspires ECRs to become independent and confident in their own skills and abilities, which is invaluable considering that the general academic system currently lacks incentives to reward such efforts. More mentors like Dorothy are needed to keep the momentum going. The open and reproducible science movement has achieved a lot over the last decade, but there is still a long way to go.
Authors:
Marta Topor, Postdoc, University of Copenhagen, Denmark
Samuel J Westwood, Lecturer in Psychology, University of Westminster, London, UK.
Emma L Henderson, Research Fellow, University of Surrey, UK
Jade S Pickering, Head of Customer Success at Gorilla Experiment Builder
Luisa Fassi, PhD Researcher, University of Cambridge, UK
Latest posts by Holly Ranger (see all)
- 2023 Culture, Employment and Development in Academic Research Survey (CEDARS) is now live! - April 11, 2023
- Articles, books, and data, and where to publish them - March 20, 2023
- NEW University Research Data Management Policy - January 18, 2023