loading page

Trustworthiness assessment as an inclusion criterion for systematic reviews -- what is the impact on results?
  • Zarko Alfirevic,
  • Jo Weeks,
  • Anna Cuthbert
Zarko Alfirevic
University of Liverpool

Corresponding Author:zarko@liv.ac.uk

Author Profile
Jo Weeks
University of Liverpool
Author Profile
Anna Cuthbert
University of Liverpool
Author Profile

Abstract

BACKGROUND There is increasing concern that a significant proportion of randomised controlled trials (RCTs) included in Cochrane reviews may not be trustworthy. Applying a trustworthiness screening tool (TST) has already had a clinically important effect on several reviews published by the Cochrane Pregnancy and Childbirth Group. OBJECTIVES We wanted to assess the impact of removing untrustworthy RCTs from already- published Cochrane reviews on a defined clinical area (ante- and post-natal nutritional interventions). METHODS We applied the tool to 18 Cochrane reviews (375 RCTs). The tool had four domains: i) is the research governance trustworthy; ii) are the baseline characteristics trustworthy; iii) is the study feasible; iv) are the results plausible?). When additional information was needed, authors were contacted using a standard template. At least two attempts were made to contact the authors. At the end of the evaluation process each study was classified as: i) included (YES to all domains); ii) excluded (retracted study); or iii) awaiting classification (any NO to the TST questions). RESULTS 95/375 studies (25%) were removed, affecting 14/18 (78%) reviews. 13/18 reviews (72%) showed a difference in the Summary of Findings tables (direction and size of effects and/or GRADE ratings). 6/18 Cochrane reviews (33%) were judged to require updating because of important differences in either in their conclusions, implication for practice, and/or implication for research. CONCLUSIONS Formal assessment of trustworthiness and inclusion only of studies that satisfy prespecified criteria for trustworthiness affect conclusions in a relatively large number of Cochrane reviews, with potentially important clinical implications for practice and research. The lack of consensus regarding the best tool(s) for assessing trustworthiness cannot be an excuse for ignoring this issue in future Cochrane reviews.
14 Sep 2023Submitted to Cochrane Evidence Synthesis and Methods
16 Sep 2023Submission Checks Completed
16 Sep 2023Assigned to Editor
20 Sep 2023Reviewer(s) Assigned
24 Sep 2023Review(s) Completed, Editorial Evaluation Pending
06 Oct 2023Editorial Decision: Revise Major
10 Nov 20231st Revision Received
10 Nov 2023Submission Checks Completed
10 Nov 2023Assigned to Editor
10 Nov 2023Review(s) Completed, Editorial Evaluation Pending
11 Nov 2023Editorial Decision: Revise Minor
14 Nov 20232nd Revision Received
16 Nov 2023Assigned to Editor
16 Nov 2023Submission Checks Completed
16 Nov 2023Review(s) Completed, Editorial Evaluation Pending
18 Nov 2023Editorial Decision: Accept