Neuron & EBRA Workshop

ERA-NET NEURON is a partner in the European Brain Council (EBC)-coordinated EU project European Brain Research Area (EBRA). EBRA was created as a catalyzing platform for brain research stakeholders to support better exploitation of the large EU investments made in brain research, achieving critical mass and economies of scale by initiating, and fostering new global research initiatives.

Within EBRA’s portfolio one task aims to broadly inform and support the research community in the implementation of future (pre-clinical) research. The combination of a strong bias toward statistically significant findings and flexibility in data analysis results in irreproducible research. In view of this reproducibility crisis in pre-clinical and clinical research it is essential to develop and implement measures for enhanced adherence to standards, e.g. ARRIVE, and 3Rs.

NEURON is fully committed to the highest possible standards of quality assurance, including working to nationally and internationally recognized best practices. NEURON’s implementation of standards in the design of animal research and full implementation of the 3Rs reflects contemporary good practice for all research using animals, and reinforces that these standards are important for ethical reasons and to obtain the best possible scientific results.

In acknowledgement of these common goals EBRA and NEURON co-organized a specific thematic workshop on the issue of transparency, reproducibility and independent verification in biomedical research on January 22^nd, 2019 in Bonn, Germany.

Within a series of keynote lectures and specific break-out sessions a number of renowned researchers focused on biomedical preclinical research, experimental design, data analysis, infrastructural support for clinical studies (ECRIN), open access to research including publication and data management plan policies, and challenges in multicenter clinical studies on TBI.

Prof. Dr. Ulrich Dirnagl as the Director of the Department of Experimental Neurology, and founding director of the QUEST – Quality, Ethics, Open Science, and Translation Center at the Berlin Institute of Health addressed in his keynote ‘Reducing bias and improving science by transforming biomedical research’ the problem of non-reproducible research findings and failure to translate bench findings into effective therapies as ‘vicious cycle of academic biomedical research’. Where scientists need to publish new, positive and spectacular results for professional advancement, journals need to publish new, positive and spectacular results to promote their Impact Factor (IF), and institutions and funders support researchers who publish new, positive and spectacular results in high IF journals. To brake this vicious cycle he suggested a number of measures: distinguishing between exploratory and confirmatory pre-clinical research, publication of NULL results, and preregistration of confirmatory pre-clinical studies to promote an open science policy (http://osf.io )in which to find, access, interoperate, and reuse (FAIR) data.

David Mellor, from the The Center for Open Science reflected in his keynote ‘Increasing impact, reproducibility, and credibility through open science’ on the mission to increase the reproducibility and transparency of science.

He introduced the Transparency and Openness Promotion (TOP) Guidelines that are promoted by the Center for Open Science.  These comprise eight policy statements for increasing the transparency and reproducibility of the published research.

Pre-registration of the study design is one of the standards defined in the TOP Guidelines. In the break-out session ‘Practical steps for implementing policies to improve study design and transparency’ it was highlighted how pre-registration helps improving the study design, mitigate the publications bias and clearly delineates exploratory from confirmatory research. In fact, pre-registration is strongly recommended for the confirmatory pre-clinical studies.

The registered reports publication format offers an approach to the scientific community that emphasizes the importance of the research question and the quality of the methodology prior to data collection. The format requires authors of empirical studies to pre-register their study protocols including hypotheses, details on the sample size and sampling methods, and the analysis plan. The submission of an empirical study outline is followed by a standard peer review process determining the quality of research methodology. If the study is accepted as a registered report, publication is granted to the authors irrespective of the outcomes. The presentation and workshop materials can be accessed here: https://osf.io/9qw53/

Prof. Dr. Christian Ohmann, former Head of the Coordination Centre for Clinical Trials (KKS) at the Medical Faculty, University Düsseldorf presented in his keynote ‘ECRIN – Facilitating European Clinical Research’ the European Clinical Research Infrastructures Network (ECRIN). ECRIN focuses on the connection of European national networks of clinical research centres and clinical trials units, and developed a sustainable infrastructure able to support the set-up, conduct, and analysis of multinational trials in Europe. ECRIN provides integrated support to multinational clinical research projects through information and consultancy, and a set of flexible services during the conduct of the project. ECRIN currently supports around 40 multinational clinical trials (mostly academic) with services and developed a series of open tools to support clinical trials. ECRIN aims to structure the landscape of clinical research via funded capacity projects.

 ‘Assessing quality of research practice: From manuscript checklists to auditing: Practical issues in preclinical study design’ Breakout Session:

Prof. Dr. Anton Bespalov, co-chair of the ECNP Preclinical Data Forum, and CEO of Partnership for Assessment and Accreditation of Scientific Practice GmbH Heidelberg (PAASP) addressed the issues of randomization, blinding, and research hypothesis generation with an engaged audience. To increase the internal validity of any preclinical study the challenges of randomization (may create selection bias), blinding interventions and outcome assessments (may create performance bias and detection bias), and predefined in and exclusion criteria (may create attrition bias) must be carefully considered and were discussed in a hands-on session.

‘Practical issues in preclinical data analysis’ Breakout Session:

Prof. Dr. Martin C. Michel, of the Pharmacology department at the Johannes Gutenberg University in Mainz (Germany) elaborated on how to reduce bias in data analysis. With the participants the issues of implementing the difference between exploratory and hypothesis-testing (confirmatory) studies, randomization and blinding which can also be applied to data analysis, and where unblinding should be done only after database lock (as in clinical trials), and the pre-specification of analysis strategies were discussed. For confirmatory (hypothesis testing) studies a scientifically plausible hypothesis exists, a pre-specification of null-hypothesis is required, and the experimental methods including sample size and analytical methods lead to statement of significance regarding the null-hypothesis. Exploratory studies can generate scientifically plausible hypotheses, do not necessarily need P-values, and have a lower prior probability. He pointed out that both types of study have a place in science, but serve different roles, and – importantly – elements of both can be combined in a single study. For instance, a study can be confirmatory for primary (and key secondary) endpoint, and exploratory for other endpoints.

 ‘Challenges in multicenter clinical studies on TBI’ Breakout Session:

Prof. Dr. Ari Ercole, a clinical anaesthetist and researcher at the University of Cambridge, and Prof. Dr. Andrew Maas, Prof Em. of Neurosurgery, Antwerp University Hospital and University of Antwerp, Belgium focused on studies in the area of traumatic brain injury (TBI). Four domains comprise epidemiological research, studies on acute, hospitalized patients, rehabilitation studies for moderate/severe TBI, and on mild TBI, concussion research, respectively. The many questions of the audience were well matched since Dr. Ercole has a particular research focus on applications of systems biology in intensive care medicine and anaesthesia after TBI and both, he and Dr. Maas are involved in CENTER-TBI (Comparative European NeuroTrauma Effectiveness Research in TBI). CENTER-TBI is a large European project that aims to better characterize TBI as a disease, and describe it in a European context, identify the most effective clinical interventions for managing TBI, and it forms part of the larger global initiative InTBIR: International Initiative for Traumatic Brain Injury Research with projects currently ongoing in Europe, the US and Canada.