Few Trials Independently Re-Analyzed
A new study has found that very few independent analyses of randomized controlled trials have actually been published.
About the Reproducibility Study
According to the Wall Street Journal, only 37 clinical trials have been re-analyzed between 1966 and 2014. A randomized controlled clinical trial is a gold standard for understanding the safety and effectiveness of a medicine, but duplicating results to confirm findings is a time consuming process (may encounter resistance over patients’ privacy and trade secrets), so the trial is usually not reproduced.
Reproducibility is a crucial step to ensure that the findings of original trials are not distorted, biased, or incomplete. The study about reproducibility, which appeared in the Journal of the American Medical Association, is apparently the first attempt to evaluate re-analyses of data from randomized controlled trials and of the changes emerging from those re-analyses.
To what extent the re-analyses may have led to wider changes in clinical practices is unclear. The authors write that, even though a re-analysis offers a contradictory message from an original article, an original article is likely to have more influence.
The study concludes that public availability of raw data should be the default for all randomized trials. This will maximize transparency about the results and conclusions of the studies. It also states that re-analyses should ideally be conducted by independent investigators (without conflicts of interests and biases) and should always be published.
Description of the Reproducibility Study
Just 37 independent studies have met their criteria for a published re-analysis from 1966 through March 2014.
- 13 re-analyses, or 35%, led to different interpretations from the original published article.
- 3 demonstrated that different patients should be treated with the medicine that was studied.
- 1 indicated that fewer patients should be treated with the medicine that was evaluated.
- 9 re-analyses, or 24%, showed that more patients should be treated with the drug.
- 31 of the re-analyses, or 84%, had overlapping authors with the original articles.
- 32 of the re-analyses, or 86%, were published by the same research group or consortium as the original articles.
- There was no significant difference in the proportion of re-analyses that led to different conclusions about who should be treated (regardless of who conducted a re-analysis).