Fri., 10/17/08 Basic Science, Paper #23, 10:22 am OTA-2008
What Authors Don’t Tell Us: The Quality of Reporting of Orthopaedic Randomized Trials Using a Checklist for Nonpharmacological Therapies
Simon Chan, MD (n); Mohit Bhandari, MD (n);
Division of Orthopaedic Surgery, McMaster University, Hamilton, Ontario, Canada
Purpose: The Consolidated Standards of Reporting Trials (CONSORT) statement for the reporting of randomized controlled trials (RCTs) has limitations in its applicability to nonpharmacological trials (NPT). In response, the Checklist to Evaluate a Report of a Nonpharmacological Trial (CLEAR NPT), a checklist that provides guidelines for the reporting of RCTs in surgery, has been developed. We aimed to (1) apply the CLEAR NPT to orthopaedic RCTs across multiple journals from 2004-2005, and (2) survey authors when items in the CLEAR NPT were not reported to determine if they were actually conducted. We hypothesized that “lack of reporting” did not necessarily correlate with “not being conducted”.
Methods: We searched for relevant orthopaedic RCTs across eight journals (four orthopaedic, four general medicine) from January 2004 through December 2005. We applied the CLEAR NPT to all eligible studies. We contacted authors to determine what methodologic safeguards were actually used, especially when details remained unclear from the publication.
Results: We included 87 RCTs from 85 scientific reports. In assessing the RCTs with the CLEAR NPT, 73 studies (84%) had unclear reporting of allocation concealment. Only 17 studies (20%) made mention of operator skill or experience. Participant, ward staff, rehabilitation staff, clinical outcome assessor, and nonclinical outcome assessor blinding were found to be unclear in 48 (55%), 63 (72%), 64 (74%), 40 (46%) and 33 studies (38%), respectively. Authors from 43 RCTs responded to our survey. In direct contact, authors reported adequate allocation concealment 41% of the time (95% confidence interval = 25%-58%) when this was unclear from the RCT report. 70% of authors acknowledged that they had set objective measures such as minimum case criteria and/or comparison to good clinical outcomes. Authors specified that they had blinded relevant groups 28% to 40% of the time, despite unclear reporting in the publications.
Conclusions: The quality of reporting in the orthopaedic literature was highly variable. Readers should not assume that bias-reducing safeguards not reported in an RCT did not occur. Our study reinforces the need for a tool like the CLEAR NPT to assess the methodology of surgical trials.
If noted, the author indicates something of value received. The codes are identified as a-research or institutional support; b-miscellaneous funding; c-royalties; d-stock options; e-consultant or employee; n-no conflicts disclosed, and *disclosure not available at time of printing.
• The FDA has not cleared this drug and/or medical device for the use described in this presentation (i.e., the drug or medical device is being discussed for an “off label” use). ◆FDA information not available at time of printing. Δ OTA Grant.