Reporting biases found in trials of cardiovascular devices
Reporting biases in published trials were first identified in 1986.1 Published randomized studies of combination chemotherapy compared with treatment with an alkylating agent as first line treatment for ovarian cancer showed a significant survival advantage for combination chemotherapy. Unpublished cancer trial registry data from the same studies, however, showed no such advantage.2 Similarly, in the treatment of multiple myeloma, registry data suggested a smaller survival advantage for combination chemotherapy (over prednisone and an alkylating agent) than the results of published studies. The author who reported the discrepancy concluded that his findings “demonstrate the value and importance of an international registry of all clinical trials.”1Subsequent evidence for biased and selective reporting included prompt or delayed publication depending on whether trial results were positive or negative3 and more favorable results and conclusions in published studies funded by industry than in those funded independently.4
The linked paper by Chang and colleagues (doi:10.1136/bmj.h2613) shows similar reporting biases in trials of medical devices.5 The authors found worrying differences between trial information submitted to the US regulator (the Food and Drug Administration) and trial information reported in medical journals. Among 177 studies of 106 high risk cardiovascular devices submitted to the FDA, fewer than half were published, and fewer than half the published studies (45%) reported primary results that precisely matched the results in submissions to the regulator. Among the published primary results, 11% (17) were judged to be “substantially different” from those submitted to the FDA. The authors concluded that “even when trials are published, the study population, primary endpoints, and results can differ substantially from data submitted to the FDA.”5
Most studies of reporting biases have examined differences in efficacy between unpublished clinical trial data and journal publication data but evidence now exists of under-reporting of adverse events. A recent BMJ editorial cites “the growing body of research on reporting biases, which documents the gross under-reporting of adverse event data in such [medical journal] sources.”6
Unfortunately, selective reporting of clinical trial data in medical journals also extends to companies’ selective non-reporting of safety data to the FDA. In 2012, the US Department of Justice announced that “GSK [GlaxoSmithKline] has agreed to plead guilty to failing to report data to the FDA and has agreed to pay a criminal fine in the amount of $242,612,800 for its unlawful conduct concerning Avandia . . . The United States alleges that, between 2001 and 2007, GSK failed to include certain safety data about Avandia, a diabetes drug, in reports to the FDA that are meant to allow the FDA to determine if a drug continues to be safe for its approved indications and to spot drug safety trends.”7
Efforts to increase the public availability of clinical trial data to prevent the serious public health consequences of overstating benefits and understating risks have triggered strong industry opposition. In 2012 the former executive director of the European Medicines Agency (EMA), Guido Rasi, committed the regulator to “proactive publication of clinical-trial data, once the marketing-authorisation process has ended.” He added “We are not here to decide if we publish clinical-trial data, but how.”8 Two pharmaceutical companies sued the EMA to prevent disclosure, and the EMA has watered down its original plans.9
Beyond adverse effects on patients of selective reporting in medical journals, the absence of publicly available data from clinical trials violates an important ethical principle of the Declaration of Helsinki: “Researchers have a duty to make publicly available the results of their research . . . Negative and inconclusive as well as positive results must be published or otherwise made publicly available.”10 Many people participate in research because they trust that the published results might improve the health of the general population.
Ignoring the Declaration of Helsinki, in 2013 the Pharmaceutical Research and Manufacturers of America (PhRMA) urged the US government to influence the European Union against the EMA’s data disclosure policy. In a letter to a US trade representative, PhRMA wrote that “Disclosure of companies’ non-public data submitted in clinical and pre-clinical dossiers and patient-level data risks damaging public health and patient welfare.”11
It is clear that the reverse is true. Non-disclosure is far more damaging. The letter of rebuttal from leaders of the high profile campaign for public registration and reporting of all trial results (AllTrials) reads,
“The world is moving towards a recognition that hiding information about what was done and what was found in clinical trials is an abuse of trial participants’ trust and exposes patients to unnecessary harm.”12
I, and many others, agree.
Cite this as: BMJ 2105;350:h2753
Simes RJ. Publication bias: the case for an international registry of clinical trials. J Clin Oncol 1986;4:1529-41.
US Department of Health and Human Services: compilation of experimental cancer therapy protocol summaries. NIH publication, Government Printing Office, 1977-1983.
Stern JM, Simes RJ. Publication bias: evidence of delayed publication in a cohort study of clinical research projects. BMJ 1997;315:640.
Lundh A, Sismondo S, Lexchin J, Busuioc OA, Bero L. Industry sponsorship and research outcome. Cochrane Database Syst Rev 2012;12:MR000033.
Chang L, Dhruva SS, Chu J, Bero LA, Redberg RF. Selective reporting in trials of high risk cardiovascular devices: cross sectional comparison between premarket approval summaries and published reports.BMJ2105:350;h2613.
Doshi P, Zito J, dosReis S.Digging for data on harms in duloxetine trials. It’s time for policy makers to get serious about drug related harms. BMJ2014;348:g3578.
Torjesin I. European Ombudsman ramps up action against European Medicines Agency over data transparency plans. BMJ2014;348:g3733.http://www.bmj.com/content/350/bmj.h2753?