Robert Schulz

Methods are one of the most valuable outputs that researchers create, however, research papers are often missing crucial details about how the research was performed, limiting reusability. Researchers can solve this by sharing reusable step-by-step protocols that describe comprehensively how a procedure was performed and citing them in publications. Reusable step-by-step protocols benefit researchers by standardizing procedures within and across research groups, increasing efficiency when training new personnel, and making it easier to re-learn protocols that have not been used recently. Protocols shared on public repositories can be used, cited, and adapted by others, potentially fostering collaborations. While some researchers mistakenly think that step-by-step protocols are only relevant to laboratory research, their value extends far beyond this. Any procedure in any field that can be described as a series of steps and repeated by others can be written as a reusable step-by-step protocol. This paper provides ten simple rules for writing a reusable step-by-step protocol, divided into three phases: prepare (Rules 1-3), write, test and refine (Rules 4-9), and share and maintain (Rule 10). The rules include guidance on how to structure a protocol, choose a documentation tool, write clear instructions that are useful to a broad audience, and use visualizations in protocols. Versioning is essential, as protocols aren’t static – the question about a protocol isn’t whether it will change, but when and how it will need to be adapted. Researchers from many fields can follow these ten simple rules to create and share reusable step-by-step protocols, enhancing the quality and efficiency of their own research while facilitating reuse and reproducibility.

The causes of the reproducibility crisis include lack of standardization and transparency in scientific reporting. Checklists such as ARRIVE and CONSORT seek to improve transparency, but they are not always followed by authors and peer review often fails to identify missing items. To address these issues, there are several automated tools that have been designed to check different rigor criteria. We have conducted a broad comparison of 11 automated tools across 9 different rigor criteria from the ScreenIT group. We found some criteria, including detecting open data, where the combination of tools showed a clear winner, a tool which performed much better than other tools. In other cases, including detection of inclusion and exclusion criteria, the combination of tools exceeded the performance of any one tool. We also identified key areas where tool developers should focus their effort to make their tool maximally useful. We conclude with a set of insights and recommendations for stakeholders in the development of rigor and transparency detection tools. The code and data for the study is available at https://github.com/PeterEckmann1/tool-comparison .

The scholarly publishing system is adapting to many changes, including open access and open data mandates, artificial intelligence, and other new technologies. Members of the research and publishing communities are working to establish a more equitable, fair, and rigorous system that serves researchers' evolving needs. Early career researchers (ECRs) are drivers of change, and publishers may wonder why and how they should involve ECRs in shaping the future of scholarly publishing. We held a virtual unconference to explore this issue with publishers and ECRs who were working to improve publishing. Some participants sought to improve peer reviewer or editor performance, whereas others sought to improve the publishing system itself through iterative or transformative change. Strategies for collaborating with ECRs to shape the future of scholarly publishing included peer review programmes, editorial programmes, ECR‐led journals, ECR boards and committee representatives, and other ECR‐initiated activities. ECRs particularly wanted to see three things improved: (1) Sharing research outputs other than publications, (2) addressing technological limitations to create systems that meet the research community's needs and facilitate knowledge advancement, and (3) fostering diversity, equity, inclusion, and accessibility. We offer tips for publishers on how to collaborate with ECRs to enhance scholarly publishing, appeal to and learn from younger researchers, and better meet researchers' needs.

The scholarly publishing system is adapting to many changes including open access and open data mandates, and new technologies, including artificial intelligence. Members of the research and publishing communities are working to establish a more equitable, fair, and rigorous system that serves researchers’ evolving needs. Early career researchers (ECRs) are drivers of change, and publishers may wonder why and how they should involve ECRs in shaping the future of scholarly publishing. We held a virtual unconference to explore this issue with publishers and ECRs who were working to improve publishing. Some participants sought to improve peer reviewer or editor performance, whereas others sought to improve or reform the publishing system itself. Strategies for achieving these goals included peer review programs, editorial programs, ECR-led journals, ECR boards and committee representatives, and other ECR-initiated activities. ECRs particularly wanted to see three things improved: 1) Sharing research outputs other than publications, 2) Addressing technological limitations to create systems that meet the research community’s needs and facilitate knowledge advancement, and 3) Fostering diversity, equity, inclusion and accessibility. We offer tips for publishers on how to collaborate with ECRs to enhance scholarly publishing, appeal to and learn from younger researchers, and better meet researchers’ needs.

Due to the high heterogeneity in outcome measures after total hip arthroplasty (THA), the prospective observational study investigated the relationships between symmetry-based (SBMs), performance-based (PBMs), and functional outcome measures in THA patients to determine necessary or redundant categories of tests. The study material consisted of 24 patients with end-stage hip osteoarthritis scheduled for THA. The patients were examined one day before surgery and consecutively on the 4th day, 9th day, and 10th week postoperatively using the SBMs (weight-bearing chair-rising test, measurements of the maximal isometric torque of the quadriceps muscle); the PBMs (10-m walk, timed up-and-go, and stair-climbing tests); and the functional outcome measure (Harris Hip Score). The results obtained in a given category of tests at different time points were compared, and the correlations between the tests were determined. The reliability of the outcome measures was determined. The results of tests in the studied categories statistically significantly (p < 0.05) improved at the 10th week postoperatively compared to preoperative results. No strong correlations were revealed between the three studied types of outcome measures in THA patients. Therefore, none of them can be considered redundant. It also means that the relevance of symmetry for a core measurement set to describe the domain function in THA patients must be further clarified.

Recent work has raised awareness about the need to replace bar graphs of continuous data with informative graphs showing the data distribution. The impact of these efforts is not known. The present observational meta-research study examined how often scientists in different fields use various graph types, and assessed whether visualization practices have changed between 2010 and 2020. We developed and validated an automated screening tool, designed to identify bar graphs of counts or proportions, bar graphs of continuous data, bar graphs with dot plots, dot plots, box plots, violin plots, histograms, pie charts, and flow charts. Papers from 23 fields (approximately 1000 papers/field per year) were randomly selected from PubMed Central and screened (n=227998). F1 scores for different graphs ranged between 0.83 and 0.95 in the internal validation set. While the tool also performed well in external validation sets, F1 scores were lower for uncommon graphs. Bar graphs are more often used incorrectly to display continuous data than they are used correctly to display counts or proportions. The proportion of papers that use bar graphs of continuous data varies markedly across fields (range in 2020: 4–58%), with high rates in biochemistry and cell biology, complementary and alternative medicine, physiology, genetics, oncology and carcinogenesis, pharmacology, microbiology and immunology. Visualization practices have improved in some fields in recent years. Fewer than 25% of papers use flow charts, which provide information about attrition and the risk of bias. The present study highlights the need for continued interventions to improve visualization and identifies fields that would benefit most.

Objectives Transparent reporting of clinical trials is essential to assess the risk of bias and translate research findings into clinical practice. While existing studies have shown that deficiencies are common, detailed empirical and field-specific data are scarce. Therefore, this study aimed to examine current clinical trial reporting and transparent research practices in sports medicine and orthopaedics. Setting Exploratory meta-research study on reporting quality and transparent research practices in orthopaedics and sports medicine clinical trials. Participants The sample included clinical trials published in the top 25% of sports medicine and orthopaedics journals over 9 months. Primary and secondary outcome measures Two independent reviewers assessed pre-registration, open data and criteria related to scientific rigour, like randomisation, blinding, and sample size calculations, as well as the study sample, and data analysis. Results The sample included 163 clinical trials from 27 journals. While the majority of trials mentioned rigour criteria, essential details were often missing. Sixty per cent (95% confidence interval (CI) 53% to 68%) of trials reported sample size calculations, but only 32% (95% CI 25% to 39%) justified the expected effect size. Few trials indicated the blinding status of all main stakeholders (4%; 95% CI 1% to 7%). Only 18% (95% CI 12% to 24%) included information on randomisation type, method and concealed allocation. Most trials reported participants’ sex/gender (95%; 95% CI 92% to 98%) and information on inclusion and exclusion criteria (78%; 95% CI 72% to 84%). Only 20% (95% CI 14% to 26%) of trials were pre-registered. No trials deposited data in open repositories. Conclusions These results will aid the sports medicine and orthopaedics community in developing tailored interventions to improve reporting. While authors typically mention blinding, randomisation and other factors, essential details are often missing. Greater acceptance of open science practices, like pre-registration and open data, is needed. As these practices have been widely encouraged, we discuss systemic interventions that may improve clinical trial reporting.

Increasing the reproducibility and trustworthiness of biomedical research requires engaging stakeholders from all levels in an institutional setting. The QUEST Center for Responsible Research aims to develop and implement new approaches to improve the culture and practice of research, tailored to the needs of these stakeholders. Members of the QUEST Center organised a brainstorm to reflect on the challenges and new opportunities encountered in implementing different projects through QUEST and share the lessons that working groups have learned over the first five years. The authors informally surveyed and interviewed working groups where relevant and highlight common themes that have influenced the success of many projects, including top-down and bottom-up engagement, managing expectations, the availability of expertise, ensuring sustainability, and considering incentives. The commentary authors conclude by encouraging the research community to view initiatives that promote reproducibility not as a one-size-fits-all undertaking, but rather as an opportunity to unite stakeholders and customise drivers of cultural change.

Recent work has raised awareness about the need to replace bar graphs of continuous data with informative graphs showing the data distribution. The impact of these efforts is not known. This observational meta-research study examined how often scientists in different fields use various graph types, and assessed whether visualization practices have changed between 2010 and 2020. We developed and validated an automated screening tool, designed to identify bar graphs of counts or proportions, bar graphs of continuous data, bar graphs with dot plots, dot plots, box plots, violin plots, histograms, pie charts, and flow charts. Papers from 23 fields (approximately 1,000 papers/field/year) were randomly selected from PubMed Central and screened (n=227,998). F1 scores for different graphs ranged between 0.83 and 0.95 in the internal validation set. While the tool also performed well in external validation sets, F1 scores were lower for uncommon graphs. Bar graphs are more often used incorrectly to display continuous data than they are used correctly to display counts or proportions. The proportion of papers that use bar graphs of continuous data varies markedly across fields (range in 2020: 4%-58%), with high rates in biochemistry and cell biology, complementary and alternative medicine, physiology, genetics, oncology and carcinogenesis, pharmacology, microbiology and immunology. Visualization practices have improved in some fields in recent years. Fewer than 25% of papers use flow charts, which provide information about attrition and the risk of bias. This study highlights the need for continued interventions to improve visualization and identifies fields that would benefit most.